如何使用node.js将数据流写入json文件

I am using node-osmosis to scrape some data from a website. I want to write the data to the json file. But it seems the data is keep comming and the file only write at the first time. I am using node-jsonfile to write data to file. By the way there is a end function when the scraping is done. But I have tried it, and if it encounter a error the done will never be called and the data is lost.

我正在使用node-osmosis从网站上抓取一些数据。我想将数据写入json文件。但似乎数据一直在进行,文件只在第一次写入。我使用node-jsonfile将数据写入文件。顺便说一下,当刮擦完成时有一个结束功能。但我已经尝试过了,如果遇到错误,就不会调用done,数据也会丢失。

osmosis
.data(function(listing) {
    jsonfile.writeFileSync(file, listing);
})

So how to write new data into the file?

那么如何将新数据写入文件?

2 个解决方案

#1

I don't know anything about osmosis, but usually '.data' is referred to a Stream when the data arrives, so you would need to create a WriteStream into the file you want the results and close it on the end callback. But as I can see in the wiki they describe it like they call the function just once... and it will contain the entire result, so maybe you are not scrapping the info you want... Maybe you need to rethink the css selectors or whatever you are using.

我对渗透性一无所知,但通常在数据到达时将'.data'引用到Stream,因此您需要在想要结果的文件中创建WriteStream并在结束回调时将其关闭。但正如我在wiki中看到的那样,他们描述它就像他们只调用一次函数一样......它将包含整个结果,所以也许你没有废弃你想要的信息......也许你需要重新考虑css选择器或者你正在使用的任何东西

Good luck!

#2

I would recommend using npm called 'jsonfile'. So basically to initially start working you'll need:

我建议使用名为'jsonfile'的npm。所以基本上最初开始工作你需要:

var jsonfile = require('jsonfile');

To read from a file:

要从文件中读取:

 jsonfile.readFile('yourJSON.json', function (err, data) {
        if (err)
            throw err;
//Your code goes here, data is that actual JSON file.
});

To Write file:

要写文件:

//data is a variable holding the data you want to write.
 jsonfile.writeFileSync('yourJSON.json', data);

You will also need body-parser as npm to parse the data.

你还需要身体解析器作为npm来解析数据。

#1