从打开的HTTP流中读取数据

I am trying to use the .NET WebRequest/WebResponse classes to access the Twitter streaming API here "http://stream.twitter.com/spritzer.json".

我正在尝试使用.NET WebRequest / WebResponse类来访问Twitter流API,这里是“http://stream.twitter.com/spritzer.json”。

I need to be able to open the connection and read data incrementally from the open connection.

我需要能够打开连接并从打开的连接中逐步读取数据。

Currently, when I call WebRequest.GetResponse method, it blocks until the entire response is downloaded. I know there is a BeginGetResponse method, but this will just do the same thing on a background thread. I need to get access to the response stream while the download is still happening. This just does not seem possible to me with these classes.

目前,当我调用WebRequest.GetResponse方法时,它会阻塞,直到整个响应被下载。我知道有一个BeginGetResponse方法,但这只会在后台线程上做同样的事情。在下载仍在进行时,我需要访问响应流。对于这些类,这对我来说似乎不太可能。

There is a specific comment about this in the Twitter documentation:

Twitter文档中有关于此的具体评论:

"Please note that some HTTP client libraries only return the response body after the connection has been closed by the server. These clients will not work for accessing the Streaming API. You must use an HTTP client that will return response data incrementally. Most robust HTTP client libraries will provide this functionality. The Apache HttpClient will handle this use case, for example."

“请注意,某些HTTP客户端库仅在服务器关闭连接后才返回响应主体。这些客户端不能用于访问Streaming API。您必须使用将以递增方式返回响应数据的HTTP客户端。最强大的HTTP客户端库将提供此功能。例如,Apache HttpClient将处理此用例。“

They point to the Appache HttpClient, but that doesn't help much because I need to use .NET.

他们指向Appache HttpClient,但这没有多大帮助,因为我需要使用.NET。

Any ideas whether this is possible with WebRequest/WebResponse, or do I have to go for lower level networking classes? Maybe there are other libraries that will allow me to do this?

是否可以使用WebRequest / WebResponse进行任何想法,或者我是否必须选择较低级别的网络类?也许有其他库可以让我这样做?

Thx Allen

4 个解决方案

#1

I ended up using a TcpClient, which works fine. Would still be interested to know if this is possible with WebRequest/WebResponse though. Here is my code in case anybody is interested:

我最终使用了TcpClient,工作正常。仍然有兴趣知道这是否可以使用WebRequest / WebResponse。这是我的代码,以防任何人感兴趣:

using (TcpClient client = new TcpClient())
{

string requestString = "GET /spritzer.json HTTP/1.1\r\n";
requestString += "Authorization: " + token + "\r\n";
requestString += "Host: stream.twitter.com\r\n";
requestString += "Connection: keep-alive\r\n";
requestString += "\r\n";

client.Connect("stream.twitter.com", 80);

using (NetworkStream stream = client.GetStream())
{
    // Send the request.
    StreamWriter writer = new StreamWriter(stream);
    writer.Write(requestString);
    writer.Flush();

    // Process the response.
    StreamReader rdr = new StreamReader(stream);

    while (!rdr.EndOfStream)
    {
        Console.WriteLine(rdr.ReadLine());
    }
}
}

#2

BeginGetResponse is the method you need. It allows you to read the response stream incrementally:

BeginGetResponse是您需要的方法。它允许您以递增方式读取响应流:

class Program
{
    static void Main(string[] args)
    {
        WebRequest request = WebRequest.Create("http://stream.twitter.com/spritzer.json");
        request.Credentials = new NetworkCredential("username", "password");
        request.BeginGetResponse(ar => 
        {
            var req = (WebRequest)ar.AsyncState;
            // TODO: Add exception handling: EndGetResponse could throw
            using (var response = req.EndGetResponse(ar))
            using (var reader = new StreamReader(response.GetResponseStream()))
            {
                // This loop goes as long as twitter is streaming
                while (!reader.EndOfStream)
                {
                    Console.WriteLine(reader.ReadLine());
                }
            }
        }, request);

        // Press Enter to stop program
        Console.ReadLine();
    }
}

Or if you feel more comfortable with WebClient (I personnally prefer it over WebRequest):

或者如果您对WebClient感觉更舒服(我个人更喜欢它而不是WebRequest):

using (var client = new WebClient())
{
    client.Credentials = new NetworkCredential("username", "password");
    client.OpenReadCompleted += (sender, e) =>
    {
        using (var reader = new StreamReader(e.Result))
        {
            while (!reader.EndOfStream)
            {
                Console.WriteLine(reader.ReadLine());
            }
        }
    };
    client.OpenReadAsync(new Uri("http://stream.twitter.com/spritzer.json"));
}
Console.ReadLine();

#3

Have you tried WebRequest.BeginGetRequestStream() ?

你试过WebRequest.BeginGetRequestStream()吗?

Or something like this:

或类似的东西:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create (http://www.twitter.com );
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream()); 

string str = reader.ReadLine();
while(str != null)
{
   Console.WriteLine(str);
   str = reader.ReadLine();
}

#4

Just use WebClient. It is designed for simple cases like this where you don't need the full power of WebRequest.

只需使用WebClient。它专为这样的简单案例而设计,您不需要WebRequest的全部功能。

System.Net.WebClient wc = new System.Net.WebClient();
Console.WriteLine(wc.DownloadString("http://stream.twitter.com/spritzer.json"));

#1