如何编写使用临时容器的范围管道？

I have a third-party function with this signature:

我有这个签名的第三方功能：

std::vector<T> f(T t);

I also have an existing potentially infinite range (of the range-v3 sort) of T named src. I want to create a pipeline that maps f to all elements of that range and flattens all the vectors into a single range with all their elements.

我还有一个名为src的T的潜在无限范围（范围-v3排序）。我想创建一个管道，将f映射到该范围的所有元素，并将所有向量展平为包含其所有元素的单个范围。

Instinctively, I would write the following.

本能地，我会写下面的内容。

 auto rng = src | view::transform(f) | view::join;

However, this won't work, because we cannot create views of temporary containers.

但是，这不起作用，因为我们无法创建临时容器的视图。

How does range-v3 support such a range pipeline?

range-v3如何支持这样的范围管道？

5 个解决方案

#1

I suspect it just can't. None of the views have any machinery to store temporaries anywhere - that's explicitly against the concept of view from the docs:

我怀疑它不能。没有任何一个视图可以在任何地方存储临时工具 - 这明显违背了文档的观点概念：

A view is a lightweight wrapper that presents a view of an underlying sequence of elements in some custom way without mutating or copying it. Views are cheap to create and copy, and have non-owning reference semantics.

视图是一个轻量级的包装器，它以某种自定义方式呈现元素的基础序列视图，而不会发生变异或复制。视图创建和复制起来很便宜，并且具有非拥有的引用语义。

So in order for that join to work and outlive the expression, something somewhere has to hold onto those temporaries. That something could be an action. This would work (demo):

因此，为了使该联接工作并且比表达式更长久，某些地方必须抓住那些临时工。这可能是一个动作。这可以工作（演示）：

auto rng = src | view::transform(f) | action::join;

except obviously not for src being infinite, and even for finite src probably adds too much overhead for you to want to use anyway.

除了显然不是src是无限的，甚至对于有限的src可能会增加太多的开销，你无论如何都要使用。

You would probably have to copy/rewrite view::join to instead use some subtly modified version of view::all (required here) that instead of requiring an lvalue container (and returning an iterator pair into it), allowed for an rvalue container that it would store internally (and returning an iterator pair into that stored version). But that's several hundred lines' worth of copying code, so seems pretty unsatisfactory, even if that works.

您可能必须复制/重写view :: join，而是使用一些精巧修改的view :: all（此处需要），而不是需要一个左值容器（并将迭代器对返回到其中），允许使用右值容器它将在内部存储（并将迭代器对返回到该存储的版本）。但这是几百行的复制代码，所以看起来非常不令人满意，即使它有效。

#2

Edited

编辑

Apparently, the code below violates the rule that views cannot own data they refer to. (However, I don't know if it's strictly forbidden to write something like this.)

显然，下面的代码违反了视图不能拥有他们引用的数据的规则。 （但是，我不知道是否严格禁止写这样的东西。）

I use ranges::view_facade to create a custom view. It holds a vector returned by f (one at a time), changing it to a range. This makes it possible to use view::join on a range of such ranges. Certainly, we can't have a random or bidirectional access to elements (but view::join itself degrades a range to an Input range), nor can we assign to them.

我使用range :: view_facade来创建自定义视图。它包含一个由f返回的向量（一次一个），将其更改为一个范围。这使得可以在一系列这样的范围上使用view :: join。当然，我们不能对元素进行随机或双向访问（但是view :: join本身会将范围降级为Input范围），我们也无法分配它们。

I copied struct MyRange from Eric Niebler's repository modifying it slightly.

我从Eric Niebler的存储库中复制了struct MyRange，稍微修改了它。

#include <iostream>
#include <range/v3/all.hpp>

using namespace ranges;

std::vector<int> f(int i) {
    return std::vector<int>(static_cast<size_t>(i), i);
}

template<typename T>
struct MyRange: ranges::view_facade<MyRange<T>> {
private:
    friend struct ranges::range_access;
    std::vector<T> data;
    struct cursor {
    private:
        typename std::vector<T>::const_iterator iter;
    public:
        cursor() = default;
        cursor(typename std::vector<T>::const_iterator it) : iter(it) {}
        T const & get() const { return *iter; }
        bool equal(cursor const &that) const { return iter == that.iter; }
        void next() { ++iter; }
        // Don't need those for an InputRange:
        // void prev() { --iter; }
        // std::ptrdiff_t distance_to(cursor const &that) const { return that.iter - iter; }
        // void advance(std::ptrdiff_t n) { iter += n; }
    };
    cursor begin_cursor() const { return {data.begin()}; }
    cursor   end_cursor() const { return {data.end()}; }
public:
    MyRange() = default;
    explicit MyRange(const std::vector<T>& v) : data(v) {}
    explicit MyRange(std::vector<T>&& v) noexcept : data (std::move(v)) {}
};

template <typename T>
MyRange<T> to_MyRange(std::vector<T> && v) {
    return MyRange<T>(std::forward<std::vector<T>>(v));
}


int main() {
    auto src = view::ints(1);        // infinite list

    auto rng = src | view::transform(f) | view::transform(to_MyRange<int>) | view::join;

    for_each(rng | view::take(42), [](int i) {
        std::cout << i << ' ';
    });
}

// Output:
// 1 2 2 3 3 3 4 4 4 4 5 5 5 5 5 6 6 6 6 6 6 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 9 9 9 9 9 9 

Compiled with gcc 5.3.0.

用gcc 5.3.0编译。

#3

range-v3 forbids views over temporary containers to help us avoid the creation of dangling iterators. Your example demonstrates exactly why this rule is necessary in view compositions:

range-v3禁止查看临时容器，以帮助我们避免创建悬空迭代器。您的示例演示了为什么在视图合成中需要此规则的原因：

auto rng = src | view::transform(f) | view::join;

If view::join were to store the begin and end iterators of the temporary vector returned by f, they would be invalidated before ever being used.

如果view :: join要存储f返回的临时向量的开始和结束迭代器，则它们在被使用之前将被无效。

"That's all great, Casey, but why don't range-v3 views store temporary ranges like this internally?"

“这一切都很棒，凯西，但为什么范围v3视图不会在内部存储像这样的临时范围？”

Because performance. Much like how the performance of the STL algorithms is predicated on the requirement that iterator operations are O(1), the performance of view compositions is predicated on the requirement that view operations are O(1). If views were to store temporary ranges in internal containers "behind your back" then the complexity of view operations - and hence compositions - would become unpredictable.

因为表现。就像STL算法的性能是如何根据迭代器操作是O（1）的要求来预测的那样，视图组合的性能取决于视图操作是O（1）的要求。如果视图是将“临时范围”存储在“背后”的内部容器中，则视图操作的复杂性 - 以及组合 - 将变得不可预测。

"Ok, fine. Given that I understand all of this wonderful design, how do I MAKE THIS WORK?!??"

“好的，很好。鉴于我理解了所有这些精彩的设计，我该如何工作？！??”

Since the view composition won't store the temporary ranges for you, you need to dump them into some kind of storage yourself, e.g.:

由于视图合成不会为您存储临时范围，您需要自己将它们转储到某种存储中，例如：

#include <iostream>
#include <vector>
#include <range/v3/range_for.hpp>
#include <range/v3/utility/functional.hpp>
#include <range/v3/view/iota.hpp>
#include <range/v3/view/join.hpp>
#include <range/v3/view/transform.hpp>

using T = int;

std::vector<T> f(T t) { return std::vector<T>(2, t); }

int main() {
    std::vector<T> buffer;
    auto store = [&buffer](std::vector<T> data) -> std::vector<T>& {
        return buffer = std::move(data);
    };

    auto rng = ranges::view::ints
        | ranges::view::transform(ranges::compose(store, f))
        | ranges::view::join;

    unsigned count = 0;
    RANGES_FOR(auto&& i, rng) {
        if (count) std::cout << ' ';
        else std::cout << '\n';
        count = (count + 1) % 8;
        std::cout << i << ',';
    }
}

Note that the correctness of this approach depends on the fact that view::join is an input range and therefore single-pass.

请注意，此方法的正确性取决于view :: join是输入范围并因此是单遍的事实。

"This isn't novice-friendly. Heck, it isn't expert-friendly. Why isn't there some kind of support for 'temporary storage materialization™' in range-v3?"

“这不是新手友好的。哎呀，它不是专家友好的。为什么在范围-v3中没有某种'临时存储物化™'的支持？”

Because we haven't gotten around to it - patches welcome ;)

因为我们还没有接触它 - 欢迎补丁;）

#4

This is another solution that doesn't require much fancy hacking. It comes at the cost of a call to std::make_shared at each call to f. But you're allocating and populating a container in f anyway, so maybe this is an acceptable cost.

这是另一种不需要太多花哨的黑客攻击的解决方案。它是以每次调用f调用std :: make_shared为代价的。但是无论如何你要在f中分配和填充容器，所以这可能是可接受的成本。

#include <range/v3/core.hpp>
#include <range/v3/view/iota.hpp>
#include <range/v3/view/transform.hpp>
#include <range/v3/view/join.hpp>
#include <vector>
#include <iostream>
#include <memory>

std::vector<int> f(int i) {
    return std::vector<int>(3u, i);
}

template <class Container>
struct shared_view : ranges::view_interface<shared_view<Container>> {
private:
    std::shared_ptr<Container const> ptr_;
public:
    shared_view() = default;
    explicit shared_view(Container &&c)
    : ptr_(std::make_shared<Container const>(std::move(c)))
    {}
    ranges::range_iterator_t<Container const> begin() const {
        return ranges::begin(*ptr_);
    }
    ranges::range_iterator_t<Container const> end() const {
        return ranges::end(*ptr_);
    }
};

struct make_shared_view_fn {
    template <class Container,
        CONCEPT_REQUIRES_(ranges::BoundedRange<Container>())>
    shared_view<std::decay_t<Container>> operator()(Container &&c) const {
        return shared_view<std::decay_t<Container>>{std::forward<Container>(c)};
    }
};

constexpr make_shared_view_fn make_shared_view{};

int main() {
    using namespace ranges;
    auto rng = view::ints | view::transform(compose(make_shared_view, f)) | view::join;
    RANGES_FOR( int i, rng ) {
        std::cout << i << '\n';
    }
}

#5

The problem here of course is the whole idea of a view - a non-storing layered lazy evaluator. To keep up with this contract, views have to pass around references to range elements, and in general they can handle both rvalue and lvalue references.

这里的问题当然是视图的整个想法 - 一个非存储的分层惰性求值器。为了跟上这个契约，视图必须传递对范围元素的引用，并且通常它们可以处理rvalue和左值引用。

Unfortunately in this specific case view::transform can only provide an rvalue reference as your function f(T t) returns a container by value, and view::join expects an lvalue as it tries to bind views (view::all) to inner containers.

不幸的是，在这个特定的情况下，view :: transform只能提供一个rvalue引用，因为你的函数f（T t）按值返回一个容器，而view :: join在尝试绑定视图（view :: all）时需要一个左值内部容器。

Possible solutions will all introduce some kind of temporary storage somewhere into the pipeline. Here are the options I came up with:

可能的解决方案都会在管道中的某处引入某种临时存储。以下是我提出的选项：

• Create a version of view::all that can internally store a container passed by an rvalue reference (As suggested by Barry). From my point of view, this violates the "non-storing view" conception and also requires some painful template coding so I would suggest against this option.
• 创建一个view :: all版本，可以在内部存储由右值引用传递的容器（如Barry所建议）。从我的角度来看，这违反了“非存储视图”概念，并且还需要一些痛苦的模板编码，因此我建议不要使用此选项。

• Use a temporary container for the whole intermediate state after the view::transform step. Can be done either by hand:

  在view :: transform步骤之后，使用临时容器作为整个中间状态。可以手工完成：

  auto rng1 = src | view::transform(f)
  vector<vector<T>> temp = rng1;
  auto rng = temp | view::join;
  

  Or using action::join. This would result in "premature evaluation", will not work with infinite src, will waste some memory, and overall has a completely different semantics from your original intention, so that is hardly a solution at all, but at least it complies with view class contracts.

  或者使用action :: join。这会导致“过早评估”，无法使用无限的src，会浪费一些内存，并且整体上与你的初衷有完全不同的语义，所以这根本不是解决方案，但至少它符合视图类合同。

• Wrap a temporary storage around the function you pass into view::transform. The simpliest example is

  在传递给view :: transform的函数周围包装临时存储。最简单的例子是

  const std::vector<T>& f_store(const T& t)
  {
    static std::vector<T> temp;
    temp = f(t);
    return temp;
  }
  

  and then pass f_store to the view::transform. As f_store returns an lvalue reference, view::join will not complain now.

  然后将f_store传递给view :: transform。当f_store返回左值引用时，view :: join现在不会抱怨。

  This of course is somewhat of a hack and will only work if you then streamline the whole range into some sink, like an output container. I believe it will withstand some straightforward transformations, like view::replace or more view::transforms, but anything more complex can try to access this temp storage in non-straightforward order.

  这当然是一种破解，只有当你将整个范围简化为某个接收器（如输出容器）时才会起作用。我相信它会经受一些简单的转换，比如view :: replace或更多view :: transforms，但是任何更复杂的转换都可以尝试以非直接的顺序访问这个临时存储。

  In that case other types of storage can be used, e.g. std::map will fix that problem and will still allow infinite src and lazy evaluation at the expense of some memory:

  在那种情况下，可以使用其他类型的存储，例如， std :: map将解决这个问题，并且仍然会以牺牲一些内存为代价来允许无限的src和延迟评估：

  const std::vector<T>& fc(const T& t)
  {
      static std::map<T, vector<T>> smap;
      smap[t] = f(t);
      return smap[t];
  }
  

  If your f function is stateless, this std::map can also be used to potentially save some calls. This approach can possibly be improved further if there is a way to guarantee that an element will no longer be required and remove it from the std::map to conserve memory. This however depends on further steps of the pipeline and the evaluation.

  如果你的f函数是无状态的，那么这个std :: map也可以用来潜在地保存一些调用。如果有一种方法可以保证不再需要一个元素并将其从std :: map中删除以节省内存，则可以进一步改进这种方法。然而，这取决于管道的进一步步骤和评估。

As these 3 solutions pretty much cover all the places to introduce temporary storage between view::transform and view::join, I think these are all the options you have. I would suggest going with #3 as it will allow you to keep the overall semantics intact and it is quite simple to implement.

由于这三个解决方案几乎涵盖了在view :: transform和view :: join之间引入临时存储的所有地方，我认为这些都是你拥有的所有选项。我建议使用＃3，因为它将允许你保持整体语义完整，并且实现起来非常简单。