Spark将JavaDStream方法转换为JavaPairDStream方法

I have this function in Spark Streaming code which splits the tweets into individual words

我在Spark流代码中有这个功能，它将tweet分解为单独的单词

JavaDStream<String> words = statuses
                .flatMap(new FlatMapFunction<String, String>() {
                    public Iterable<String> call(String in) {
                        return Arrays.asList(in.split(" "));
                    }
                });

I need to modify it so that it returns the words and the original tweet against each word. I have tried the below, But I am getting the java.lang.ClassCastException: scala.Tuple2 cannot be cast to java.lang.Iterable error during run time.

我需要对它进行修改，以便它针对每个单词返回单词和原始tweet。我尝试过下面的方法，但是我得到了java.lang。ClassCastException:scala。Tuple2不能被转换为java.lang。在运行时可重复的错误。

    JavaPairDStream<String, String> wordTweets = statuses.flatMapToPair(
            new PairFlatMapFunction<String, String, String>() {
                public Iterable<Tuple2<String, String>> call(String in){
                    Tuple2<String, String> tuple2 = new Tuple2(Arrays.asList(in.split(" ")), in);
                    return (Iterable<Tuple2<String, String>>) tuple2;
                }
            });

1 个解决方案

#1

Use the following to get the functionality

使用以下代码获取功能

JavaPairDStream<String, String> locationspairRdd=OutStreamRDD.mapToPair(new PairFunction<String, String, String>() {
    public Tuple2<String, String> call(String arg0) throws Exception {
        return null;
    }
});