I'm building a program that has a class used locally, but I want the same class to be used the same way over the network. This means I need to be able to make synchronous calls to any of its public methods. The class reads and writes files, so I think XML-RPC is too much overhead. I created a basic rpc client/server using the examples from twisted, but I'm having trouble with the client.
我正在构建一个具有本地使用类的程序,但我希望通过网络以相同的方式使用相同的类。这意味着我需要能够对其任何公共方法进行同步调用。该类读写文件,因此我认为XML-RPC开销太大。我使用twisted中的示例创建了一个基本的rpc客户端/服务器,但是我遇到了客户端问题。
c = ClientCreator(reactor, Greeter)
c.connectTCP(self.host, self.port).addCallback(request)
reactor.run()
This works for a single call, when the data is received I'm calling reactor.stop(), but if I make any more calls the reactor won't restart. Is there something else I should be using for this? maybe a different twisted module or another framework?
这适用于单个调用,当收到数据时我正在调用reactor.stop(),但是如果我再进行一次调用,反应器将不会重新启动。我还应该使用其他东西吗?也许是一个不同的扭曲模块或其他框架?
(I'm not including the details of how the protocol works, because the main point is that I only get one call out of this.)
(我不包括协议如何工作的细节,因为重点是我只能接到一个呼叫。)
Addendum & Clarification:
附录和说明:
I shared a google doc with notes on what I'm doing. http://docs.google.com/Doc?id=ddv9rsfd_37ftshgpgz
我与谷歌文档分享了我正在做的事情。 http://docs.google.com/Doc?id=ddv9rsfd_37ftshgpgz
I have a version written that uses fuse and can combine multiple local folders into the fuse mount point. The file access is already handled within a class, so I want to have servers that give me network access to the same class. After continuing to search, I suspect pyro (http://pyro.sourceforge.net/) might be what I'm really looking for (simply based on reading their home page right now) but I'm open to any suggestions.
我有一个使用保险丝的版本,可以将多个本地文件夹组合到保险丝安装点。文件访问已经在一个类中处理,所以我想让服务器给我网络访问同一个类。在继续搜索之后,我怀疑pyro(http://pyro.sourceforge.net/)可能是我真正想要的东西(仅仅基于现在阅读他们的主页)但我对任何建议持开放态度。
I could achieve similar results by using an nfs mount and combining it with my local folder, but I want all of the peers to have access to the same combined filesystem, so that would require every computer to bee an nfs server with a number of nfs mounts equal to the number of computers in the network.
我可以通过使用nfs mount并将其与我的本地文件夹组合来实现类似的结果,但我希望所有的对等体都能访问相同的组合文件系统,因此这将要求每台计算机都具有nfs服务器的nfs服务器安装数等于网络中的计算机数量。
Conclusion: I have decided to use rpyc as it gave me exactly what I was looking for. A server that keeps an instance of a class that I can manipulate as if it was local. If anyone is interested I put my project up on Launchpad (http://launchpad.net/dstorage).
结论:我决定使用rpyc,因为它给了我正是我想要的东西。一个服务器,它保存一个我可以操作的类的实例,就好像它是本地的一样。如果有人有兴趣我将我的项目放在Launchpad(http://launchpad.net/dstorage)上。
4 个解决方案
#1
2
If you're even considering Pyro, check out RPyC first, and re-consider XML-RPC.
如果您甚至考虑使用Pyro,请首先查看RPyC,然后重新考虑XML-RPC。
Regarding Twisted: try leaving the reactor up instead of stopping it, and just ClientCreator(...).connectTCP(...) each time.
关于Twisted:尝试离开反应堆而不是停止它,每次只需ClientCreator(...)。connectTCP(...)。
If you self.transport.loseConnection() in your Protocol you won't be leaving open connections.
如果您在协议中使用self.transport.loseConnection(),则不会保留打开的连接。
#2
2
Why do you feel that it needs to be synchronous?
为什么你觉得它需要同步?
If you want to ensure that only one of these is happening at a time, invoke all of the calls through a DeferredSemaphore so you can rate limit the actual invocations (to any arbitrary value).
如果要确保一次只发生其中一个,请通过DeferredSemaphore调用所有调用,以便对实际调用(对任意值)进行速率限制。
If you want to be able to run multiple streams of these at different times, but don't care about concurrency limits, then you should at least separate reactor startup and teardown from the invocations (the reactor should run throughout the entire lifetime of the process).
如果您希望能够在不同时间运行多个这些流,但不关心并发限制,那么您应该至少将反应器启动和拆除与调用分开(反应器应该在整个过程的整个生命周期内运行) )。
If you just can't figure out how to express your application's logic in a reactor pattern, you can use deferToThread and write a chunk of purely synchronous code -- although I would guess this would not be necessary.
如果您无法弄清楚如何在反应器模式中表达应用程序的逻辑,您可以使用deferToThread并编写一大块纯同步代码 - 尽管我猜这不是必需的。
#3
1
For a synchronous client, Twisted probably isn't the right option. Instead, you might want to use the socket module directly.
对于同步客户端,Twisted可能不是正确的选项。相反,您可能希望直接使用套接字模块。
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((self.host, self.port))
s.send(output)
data = s.recv(size)
s.close()
The recv() call might need to be repeated until you get an empty string, but this shows the basics.
recv()调用可能需要重复,直到你得到一个空字符串,但这显示了基础知识。
Alternatively, you can rearrange your entire program to support asynchronous calls...
或者,您可以重新排列整个程序以支持异步调用...
#4
1
If you are using Twisted you should probably know that:
如果你使用Twisted,你应该知道:
- You will not be making synchronous calls to any network service
- The reactor can only ever be run once, so do not stop it (by calling
reactor.stop()) until your application is ready to exit.
您不会同步调用任何网络服务
反应堆只能运行一次,所以不要停止它(通过调用reactor.stop())直到你的应用程序准备好退出。
I hope this answers your question. I personally believe that Twisted is exactly the correct solution for your use case, but that you need to work around your synchronicity issue.
我希望这回答了你的问题。我个人认为Twisted是您用例的正确解决方案,但您需要解决同步性问题。
Addendum & Clarification:
附录和说明:
Part of what I don't understand is that when I call reactor.run() it seems to go into a loop that just watches for network activity. How do I continue running the rest of my program while it uses the network? if I can get past that, then I can probably work through the synchronicity issue.
我不明白的一部分是,当我调用reactor.run()时,它似乎进入一个只监视网络活动的循环。如何在使用网络时继续运行程序的其余部分?如果我能够解决这个问题,那么我可能会解决同步问题。
That is exactly what reactor.run() does. It runs a main loop which is an event reactor. It will not only wait for entwork events, but anything else you have scheduled to happen. With Twisted you will need to structure the rest of your application in a way to deal with its asynchronous nature. Perhaps if we knew what kind of application it is, we could advise.
这正是reactor.run()的作用。它运行一个主循环,它是一个事件反应器。它不仅会等待诱人的活动,还会等待您计划发生的任何事情。使用Twisted,您将需要以处理其异步性质的方式构建应用程序的其余部分。也许如果我们知道它是什么样的应用程序,我们可以提供建议。
#1
2
If you're even considering Pyro, check out RPyC first, and re-consider XML-RPC.
如果您甚至考虑使用Pyro,请首先查看RPyC,然后重新考虑XML-RPC。
Regarding Twisted: try leaving the reactor up instead of stopping it, and just ClientCreator(...).connectTCP(...) each time.
关于Twisted:尝试离开反应堆而不是停止它,每次只需ClientCreator(...)。connectTCP(...)。
If you self.transport.loseConnection() in your Protocol you won't be leaving open connections.
如果您在协议中使用self.transport.loseConnection(),则不会保留打开的连接。
#2
2
Why do you feel that it needs to be synchronous?
为什么你觉得它需要同步?
If you want to ensure that only one of these is happening at a time, invoke all of the calls through a DeferredSemaphore so you can rate limit the actual invocations (to any arbitrary value).
如果要确保一次只发生其中一个,请通过DeferredSemaphore调用所有调用,以便对实际调用(对任意值)进行速率限制。
If you want to be able to run multiple streams of these at different times, but don't care about concurrency limits, then you should at least separate reactor startup and teardown from the invocations (the reactor should run throughout the entire lifetime of the process).
如果您希望能够在不同时间运行多个这些流,但不关心并发限制,那么您应该至少将反应器启动和拆除与调用分开(反应器应该在整个过程的整个生命周期内运行) )。
If you just can't figure out how to express your application's logic in a reactor pattern, you can use deferToThread and write a chunk of purely synchronous code -- although I would guess this would not be necessary.
如果您无法弄清楚如何在反应器模式中表达应用程序的逻辑,您可以使用deferToThread并编写一大块纯同步代码 - 尽管我猜这不是必需的。
#3
1
For a synchronous client, Twisted probably isn't the right option. Instead, you might want to use the socket module directly.
对于同步客户端,Twisted可能不是正确的选项。相反,您可能希望直接使用套接字模块。
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((self.host, self.port))
s.send(output)
data = s.recv(size)
s.close()
The recv() call might need to be repeated until you get an empty string, but this shows the basics.
recv()调用可能需要重复,直到你得到一个空字符串,但这显示了基础知识。
Alternatively, you can rearrange your entire program to support asynchronous calls...
或者,您可以重新排列整个程序以支持异步调用...
#4
1
If you are using Twisted you should probably know that:
如果你使用Twisted,你应该知道:
- You will not be making synchronous calls to any network service
- The reactor can only ever be run once, so do not stop it (by calling
reactor.stop()) until your application is ready to exit.
您不会同步调用任何网络服务
反应堆只能运行一次,所以不要停止它(通过调用reactor.stop())直到你的应用程序准备好退出。
I hope this answers your question. I personally believe that Twisted is exactly the correct solution for your use case, but that you need to work around your synchronicity issue.
我希望这回答了你的问题。我个人认为Twisted是您用例的正确解决方案,但您需要解决同步性问题。
Addendum & Clarification:
附录和说明:
Part of what I don't understand is that when I call reactor.run() it seems to go into a loop that just watches for network activity. How do I continue running the rest of my program while it uses the network? if I can get past that, then I can probably work through the synchronicity issue.
我不明白的一部分是,当我调用reactor.run()时,它似乎进入一个只监视网络活动的循环。如何在使用网络时继续运行程序的其余部分?如果我能够解决这个问题,那么我可能会解决同步问题。
That is exactly what reactor.run() does. It runs a main loop which is an event reactor. It will not only wait for entwork events, but anything else you have scheduled to happen. With Twisted you will need to structure the rest of your application in a way to deal with its asynchronous nature. Perhaps if we knew what kind of application it is, we could advise.
这正是reactor.run()的作用。它运行一个主循环,它是一个事件反应器。它不仅会等待诱人的活动,还会等待您计划发生的任何事情。使用Twisted,您将需要以处理其异步性质的方式构建应用程序的其余部分。也许如果我们知道它是什么样的应用程序,我们可以提供建议。