是否对数组（结构类型）进行了优化以避免不必要的结构值复制？

For memory performance reasons I have an array of structures since the number of items is large and the items get tossed regularly and hence thrashing the GC heap. This is not a question of whether I should use large structures; I have already determined GC trashing is causing performance problems. My question is when I need to process this array of structures, should I avoid using LINQ? Since the structure is not small it is not wise to pass it around by value, and I have no idea if the LINQ code generator is smart enough to do this or not. The structure looks like this:

出于内存性能的原因，我有一个结构数组，因为项目数量很大，而且项目会被定期抛出，从而导致GC堆抖动。这不是我是否应该使用大型结构的问题;我已经确定GC废物导致性能问题。我的问题是当我需要处理这个结构数组时，我应该避免使用LINQ吗？由于结构不小，所以通过值传递它是不明智的，我不知道LINQ代码生成器是否足够智能来执行此操作。结构如下所示：

public struct ManufacturerValue
{
    public int ManufacturerID;
    public string Name;
    public string CustomSlug;
    public string Title;
    public string Description;
    public string Image;
    public string SearchFilters;
    public int TopZoneProduction;
    public int TopZoneTesting;
    public int ActiveProducts;
}

So let's say we have an array of these values and I want to extract a dictionary of custom slugs to manufacturers ID's. Before I changed this to a structure it was a class, so the original code was written with a simple LINQ query:

因此，假设我们有一系列这些值，我想向制造商ID提取自定义slu的字典。在我将其更改为结构之前，它是一个类，因此原始代码是使用简单的LINQ查询编写的：

ManufacturerValue[] = GetManufacturerValues();
var dict = values.Where(p => !string.IsNullOrEmpty(p.CustomSlug))
                 .ToDictionary(p => p.CustomSlug, p => p.ManufacturerID);

My concern is I want to understand how LINQ is going to generate the actual code to build this dictionary. My suspicion is that internally the LINQ code is going to end up something like this naive implementation:

我担心的是我想了解LINQ将如何生成构建此字典的实际代码。我怀疑LINQ代码内部会出现类似这种天真的实现：

var dict = new Dictionary<string, int>();
for (var i = 0; i < values.Length; i++) {
    var value = values[i];
    if (!string.IsNullOrEmpty(value.CustomSlug)) {
        dict.Add(value.CustomSlug, value.ManufacturerID);
    }
}

which would be bad, because the third line is going to create a local copy of the structure, which will be slow because the structure is large and will instead thrash the memory bus. We also do not need anything but the ID and custom slug from it so it will copy a lot of useless information on every iteration. Rather if I coded it efficiently myself, I would write it like this:

这将是不好的，因为第三行将创建一个结构的本地副本，这将是缓慢的，因为结构很大，而将颠倒内存总线。除了ID和自定义slug之外，我们也不需要任何东西，所以它会在每次迭代时复制很多无用的信息。相反，如果我自己有效编码，我会这样写：

var dict = new Dictionary<string, int>();
for (var i = 0; i < values.Length; i++) {
    if (!string.IsNullOrEmpty(values[i].CustomSlug)) {
        dict.Add(values[i].CustomSlug, values[i].ManufacturerID);
    }
}

So does anyone know if the code generator is smart enough to use simple array indexing like the second example when generator code to run over arrays of structures, or will it implement the more naive but slower first implementation?

那么，有没有人知道代码生成器是否足够智能以使用简单的数组索引，如第二个示例，当生成器代码在结构数组上运行时，或者它是否会实现更天真但更慢的第一个实现？

What is the best way to decompile this kind of code to find out what the code generator would actually do for this?

反编译这种代码以找出代码生成器实际上会为此做什么的最佳方法是什么？

UPDATE

UPDATE

These changes are now in production. As it turns out in the process of re-writing the code and using the Dot Memory profiler to identify how much memory was being used and where, I found two memory leaks in the Phalanger PHP compiler code. That was one of the reasons the amount of memory our processes were using kept growing, and one of the memory leaks was really nasty and actually caused by the Microsoft Async code (probably worth a blog or a stack overflow question/answer to help others avoid it).

这些变化现在正在生产中。事实证明，在重写代码和使用Dot Memory分析器来识别正在使用多少内存以及在哪里，我发现Phalanger PHP编译器代码中存在两个内存泄漏。这是我们的进程使用的内存量持续增长的原因之一，其中一个内存泄漏真的很讨厌，实际上是由Microsoft异步代码引起的（可能值得博客或堆栈溢出问题/答案，以帮助其他人避免它）。

Anyway, once I found the memory leaks and fixed them I pushed that code live without any of the memory optimizations to convert from classes to structures, and oddly enough this actually caused the GC to thrash even more. I was seeing periods of time when the GC would be using up to 27% of the CPU according to the performance counters. Most likely these big blocks were previously not getting GC'ed due to the memory leaks, so they simply hung around. Once the code was fixed the GC started behaving even worse than before.

无论如何，一旦我发现内存泄漏并修复它们，我就推送了代码，没有任何内存优化来从类转换为结构，奇怪的是这实际上导致了GC更多地捶打。根据性能计数器，我看到GC将使用高达27％的CPU的时间段。很可能这些大块由于内存泄漏而以前没有得到GC，所以他们只是挂了。一旦代码被修复，GC开始表现得比以前更糟糕。

Finally we finished up the code to convert these classes to structures using the feedback in this question, and now our total memory usage at peak is about 50% of what it was, it rapidly drops down when the load on the server goes away and more importantly we are seeing only 0.05% of the CPU being used for GC, if even that. So if anyone is wondering whether these changes can have an impact on the real world, they really can, especially if you have objects that normally hang around for a while so get stuck in the 2nd gen heap and then need to get tossed and garbage collected.

最后，我们完成了使用此问题中的反馈将这些类转换为结构的代码，现在我们在峰值时的总内存使用量大约是它的50％，当服务器上的负载消失时它会迅速下降重要的是，我们看到只有0.05％的CPU用于GC，即便如此。因此，如果有人想知道这些变化是否会对现实世界产生影响，他们真的可以，特别是如果你有一些物品通常会闲置一段时间，所以卡在第二代堆中，然后需要被扔掉并收集垃圾。

5 个解决方案

#1

What is the best way to decompile this kind of code to find out what the code generator would actually do for this?

反编译这种代码以找出代码生成器实际上会为此做什么的最佳方法是什么？

There is no need to decompile the code. All LINQ to Objects method implementation can be seen at Reference Source.

无需反编译代码。所有LINQ to Objects方法实现都可以在Reference Source中看到。

Regarding your concrete question. You can expect a lot of struct copy operations when using LINQ (and in general IEnumerable<T> and Func<T, ..> based methods).

关于你的具体问题。在使用LINQ（通常是基于IEnumerable 和Func 的方法）时，您可以期待很多结构复制操作。，..>

For instance, the current element of IEnumerator<T> is accessed via property Current defined as follows

例如，IEnumerator 的当前元素是通过如下定义的属性Current来访问的

T Current { get; }

so accessing at least involves one copy. But enumerator implementations usually store the current element into a field during the MoveNext method, so I would say you can safely count 2 copy operations.

所以访问至少涉及一个副本。但是枚举器实现通常在MoveNext方法期间将当前元素存储到字段中，所以我想你可以安全地计算2个复制操作。

And of course, every Func<T, ...> will cause another copy because T is input argument.

当然，每个Func 都会导致另一个副本，因为T是输入参数。，...>

So in general you should avoid LINQ in such scenarios.

所以一般来说，在这种情况下你应该避免使用LINQ。

Or, you can use the old school technique of simulating reference via array and index. So instead of this:

或者，您可以使用通过数组和索引模拟引用的旧学校技术。所以不是这样的：

var dict = values
    .Where(p => !string.IsNullOrEmpty(p.CustomSlug))
    .ToDictionary(p => p.CustomSlug, p => p.ManufacturerID);

you can avoid struct copy by using this:

你可以使用这个来避免struct copy：

var dict = Enumerable.Range(0, values.Length)
    .Where(i => !string.IsNullOrEmpty(values[i].CustomSlug))
    .ToDictionary(i => values[i].CustomSlug, i => values[i].ManufacturerID);

UPDATE: Since seems like there is an interest to the subject, I'll provide you a variation of the last technique which can make your life easier still avoiding the excessive struct copy.

更新：由于似乎对主题感兴趣，我将为您提供最后一种技术的变体，它可以使您的生活更轻松，同时避免过多的结构复制。

Let say your ManufacturerValue was a class and you have used a lot of LINQ queries like the one in the example. Then you switched to a struct.

假设您的ManufacturerValue是一个类，并且您使用了许多LINQ查询，例如示例中的查询。然后你切换到结构。

You can also create a wrapper struct and helper extension method like this

您还可以像这样创建包装器结构和辅助扩展方法

public struct ManufacturerValue
{
    public int ManufacturerID;
    public string Name;
    public string CustomSlug;
    public string Title;
    public string Description;
    public string Image;
    public string SearchFilters;
    public int TopZoneProduction;
    public int TopZoneTesting;
    public int ActiveProducts;
}

public struct ManufacturerValueRef
{
    public readonly ManufacturerValue[] Source;
    public readonly int Index;
    public ManufacturerValueRef(ManufacturerValue[] source, int index) { Source = source; Index = index; }
    public int ManufacturerID => Source[Index].ManufacturerID;
    public string Name => Source[Index].Name;
    public string CustomSlug => Source[Index].CustomSlug;
    public string Title => Source[Index].Title;
    public string Description => Source[Index].Description;
    public string Image => Source[Index].Image;
    public string SearchFilters => Source[Index].SearchFilters;
    public int TopZoneProduction => Source[Index].TopZoneProduction;
    public int TopZoneTesting => Source[Index].TopZoneTesting;
    public int ActiveProducts => Source[Index].ActiveProducts;
}

public static partial class Utils
{
    public static IEnumerable<ManufacturerValueRef> AsRef(this ManufacturerValue[] values)
    {
        for (int i = 0; i < values.Length; i++)
            yield return new ManufacturerValueRef(values, i);
    }
}

It's additional (one time) effort, but with the following benefits:

这是额外的（一次）努力，但具有以下好处：

(1) It's a struct, but with a fixed size, so the copy overhead will be negligible compared to normal reference (one additional int).
(2) You can extend the actual data struct size w/o worry.
(3) All you need to do with your LINQ queries is to add .AsRef()

（1）它是一个结构体，但是具有固定的大小，因此与正常引用（一个额外的int）相比，复制开销可以忽略不计。（2）您可以无需扩展实际的数据结构大小。（3）你需要对LINQ查询做的只是添加.AsRef（）

Sample:

样品：

var dict = values.AsRef()
    .Where(p => !string.IsNullOrEmpty(p.CustomSlug))
    .ToDictionary(p => p.CustomSlug, p => p.ManufacturerID);

#2

Structs are [pass by value][1] - so I'm fairly certain just the act of using delegates for your ToDictionary will result in two copies, regardless of what else is going on.

结构是[通过值传递] [1] - 所以我很确定只使用代理为您的ToDictionary行为将导致两个副本，无论其他内容是什么。

In other words, consider

换句话说，考虑一下

.ToDictionary(p => p.CustomSlug, p => p.ManufacturerID);

As equivalent to:

相当于：

var key = GetKey(values[i]);
var value = GetValue(values[i]);

.ToDictionary(key, value);

which obviously creates two copies of the struct to pass to GetKey and GetValue.

这显然创建了两个结构副本以传递给GetKey和GetValue。

#3

If you need to relax the garbage collector a bit, you may want to use gcServer option in your app.config file:

如果您需要稍微放松一下垃圾收集器，您可能需要在app.config文件中使用gcServer选项：

<configuration>
    <runtime>
        <gcServer enabled="true" />
    </runtime>
</configuration>

To see what kind of IL is generated based on your LINQ code, LinqPad is a great tool.

要查看基于LINQ代码生成的IL类型，LinqPad是一个很棒的工具。

Unfortunately, I dont have any clue about using LINQ against enumeration of structs. I am usually using structs to keep a little amount of value type.

不幸的是，我对使用LINQ反对结构的枚举没有任何线索。我通常使用结构来保留一些值类型。

Maybe relaxing the GC will help you to circumvent your performance issue, and give classes another chance ? I also have an application that do a massive amount of object creation and disposal where the performances were plagged by the GC Frenzy. Using GCServer="true" solved it, in exchange of a sligh increase of used private memory.

也许放松GC可以帮助您规避性能问题，并为课程提供另一次机会？我还有一个应用程序可以进行大量的对象创建和处理，其中表演由GC Frenzy设置。使用GCServer =“true”解决了它，以换取使用私有内存的轻微增加。

#4

The arrows:

箭头：

p => !string.IsNullOrEmpty(p.CustomSlug)
p => p.CustomSlug
p => p.ManufacturerID

are each compiled into an actual method where p is a value parameter of the method. Those methods are then passed to Linq in the form of Func delegate instances. Since they are value parameters, your struct is passed by value.

每个都被编译成一个实际的方法，其中p是该方法的值参数。然后，这些方法以Func委托实例的形式传递给Linq。由于它们是值参数，因此您的结构将按值传递。

Maybe you can use:

也许你可以使用：

ManufacturerValue[] values = GetManufacturerValues();
var dict = Enumerate.Range(0, values.Length)
  .Where(i => !string.IsNullOrEmpty(values[i].CustomSlug))
  .ToDictionary(i => values[i].CustomSlug, i => values[i].ManufacturerID);

This just captures the array reference in each of the lambda arrows (closures).

这只是捕获每个lambda箭头（闭包）中的数组引用。

Edit: I had not seen that Ivan Stoev's answer already had this suggestion. Upvote his answer instead.

编辑：我没有看到Ivan Stoev的回答已经有了这个建议。反而回答他的回答。

#5

I have benchmarked the performance of Linq's Where() on 10 million structs vs classes large and small.

我已经对Linq的Where（）在1000万个结构与大小的类别上的性能进行了基准测试。

Structs are faster in all cases.

在所有情况下，结构都更快。

code: https://github.com/Erikvv/linq-large-struct-benchmark

代码：https：//github.com/Erikvv/linq-large-struct-benchmark

#1