一个mock / stub python模块如何像urllib一样

I need to test a function that needs to query a page on an external server using urllib.urlopen (it also uses urllib.urlencode). The server could be down, the page could change; I can't rely on it for a test.

我需要测试一个需要使用urllib.urlopen在外部服务器上查询页面的函数（它还使用urllib.urlencode）。服务器可能已关闭，页面可能会更改;我不能依靠它进行测试。

What is the best way to control what urllib.urlopen returns?

控制urllib.urlopen返回的最佳方法是什么？

7 个解决方案

#1

Another simple approach is to have your test override urllib's urlopen() function. For example, if your module has

另一个简单的方法是让你的测试覆盖urllib的urlopen（）函数。例如，如果您的模块有

import urllib

def some_function_that_uses_urllib():
    ...
    urllib.urlopen()
    ...

You could define your test like this:

您可以像这样定义测试：

import mymodule

def dummy_urlopen(url):
    ...

mymodule.urllib.urlopen = dummy_urlopen

Then, when your tests invoke functions in mymodule, dummy_urlopen() will be called instead of the real urlopen(). Dynamic languages like Python make it super easy to stub out methods and classes for testing.

然后，当你的测试调用mymodule中的函数时，将调用dummy_urlopen（）而不是真正的urlopen（）。像Python这样的动态语言使得用于测试的方法和类非常容易。

See my blog posts at http://softwarecorner.wordpress.com/ for more information about stubbing out dependencies for tests.

有关存根测试依赖关系的更多信息，请参阅http://softwarecorner.wordpress.com/上的博客文章。

#2

I am using Mock's patch decorator:

我正在使用Mock的补丁装饰器：

from mock import patch

[...]

@patch('urllib.urlopen')
def test_foo(self, urlopen_mock):
    urlopen_mock.return_value = MyUrlOpenMock()

#3

Did you give Mox a look? It should do everything you need. Here is a simple interactive session illustrating the solution you need:

你有没有看过Mox？它应该做你需要的一切。这是一个简单的交互式会话，说明您需要的解决方案：

>>> import urllib
>>> # check that it works
>>> urllib.urlopen('http://www.google.com/')
<addinfourl at 3082723820L ...>
>>> # check what happens when it doesn't
>>> urllib.urlopen('http://hopefully.doesnotexist.com/')
#-- snip --
IOError: [Errno socket error] (-2, 'Name or service not known')

>>> # OK, let's mock it up
>>> import mox
>>> m = mox.Mox()
>>> m.StubOutWithMock(urllib, 'urlopen')
>>> # We can be verbose if we want to :)
>>> urllib.urlopen(mox.IgnoreArg()).AndRaise(
...   IOError('socket error', (-2, 'Name or service not known')))

>>> # Let's check if it works
>>> m.ReplayAll()
>>> urllib.urlopen('http://www.google.com/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site-packages/mox.py", line 568, in __call__
    raise expected_method._exception
IOError: [Errno socket error] (-2, 'Name or service not known')

>>> # yay! now unset everything
>>> m.UnsetStubs()
>>> m.VerifyAll()
>>> # and check that it still works
>>> urllib.urlopen('http://www.google.com/')
<addinfourl at 3076773548L ...>

#4

HTTPretty works in the exact same way that FakeWeb does. HTTPretty works in the socket layer, so it should work intercepting any python http client libraries. It's battle tested against urllib2, httplib2 and requests

HTTPretty的工作方式与FakeWeb完全相同。 HTTPretty在套接字层中工作，因此它应该可以拦截任何python http客户端库。它针对urllib2，httplib2和请求进行了战斗测试

import urllib2
from httpretty import HTTPretty, httprettified


@httprettified
def test_one():
    HTTPretty.register_uri(HTTPretty.GET, "http://yipit.com/",
                           body="Find the best daily deals")

    fd = urllib2.urlopen('http://yipit.com')
    got = fd.read()
    fd.close()

    assert got == "Find the best daily deals"

#5

Probably the best way to handle this is to split up the code, so that logic that processes the page contents is split from the code that fetches the page.

处理此问题的最佳方法可能是拆分代码，以便处理页面内容的逻辑从获取页面的代码中分离出来。

Then pass an instance of the fetcher code into the processing logic, then you can easily replace it with a mock fetcher for the unit test.

然后将fetcher代码的实例传递给处理逻辑，然后您可以使用模拟获取器轻松替换它以进行单元测试。

e.g.

例如

class Processor(oject):
    def __init__(self, fetcher):
        self.m_fetcher = fetcher

    def doProcessing(self):
        ## use self.m_fetcher to get page contents

class RealFetcher(object):
    def fetchPage(self, url):
        ## get real contents

class FakeFetcher(object):
    def fetchPage(self, url):
        ## Return whatever fake contents are required for this test

#6

In case you don't want to even load the module:

如果您不想加载模块：

import sys,types
class MockCallable():
  """ Mocks a function, can be enquired on how many calls it received """
  def __init__(self, result):
    self.result  = result
    self._calls  = []

  def __call__(self, *arguments):
    """Mock callable"""
    self._calls.append(arguments)
    return self.result

  def called(self):
    """docstring for called"""
    return self._calls

class StubModule(types.ModuleType, object):
  """ Uses a stub instead of loading libraries """

  def __init__(self, moduleName):
    self.__name__ = moduleName
    sys.modules[moduleName] = self

  def __repr__(self):
    name  = self.__name__
    mocks = ', '.join(set(dir(self)) - set(['__name__']))
    return "<StubModule: %(name)s; mocks: %(mocks)s>" % locals()

class StubObject(object):
  pass

And then:

接着：

>>> urllib = StubModule("urllib")
>>> import urllib # won't actually load urllib

>>> urls.urlopen = MockCallable(StubObject())

>>> example = urllib.urlopen('http://example.com')
>>> example.read = MockCallable('foo')

>>> print(example.read())
'foo'

#7

The simplest way is to change your function so that it doesn't necessarily use urllib.urlopen. Let's say this is your original function:

最简单的方法是更改您的函数，以便它不一定使用urllib.urlopen。让我们说这是你原来的功能：

def my_grabber(arg1, arg2, arg3):
    # .. do some stuff ..
    url = make_url_somehow()
    data = urllib.urlopen(url)
    # .. do something with data ..
    return answer

Add an argument which is the function to use to open the URL. Then you can provide a mock function to do whatever you need:

添加一个参数，该参数是用于打开URL的函数。然后你可以提供一个模拟函数来做你需要的任何事情：

def my_grabber(arg1, arg2, arg3, urlopen=urllib.urlopen):
    # .. do some stuff ..
    url = make_url_somehow()
    data = urlopen(url)
    # .. do something with data ..
    return answer

def test_my_grabber():
    my_grabber(arg1, arg2, arg3, urlopen=my_mock_open)

#1