pip和easy_install配置好了之后，准备安装scrapy
第一部分
执行pip install scrapy
You are using pip version 6.0.8, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Downloading cryptography-2.0.3.tar.gz (427kB)
5% |# | 24kB 10kB/s eta 0:00:39 Exception:
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\pip\_vendor\requests\packages\urllib3\re
ponse.py", line 186, in read
data = self._fp.read(amt)
File "C:\Python34\lib\site-packages\pip\_vendor\cachecontrol\filewrapper.py"
line 49, in read
data = self.__fp.read(amt)
File "C:\Python34\lib\http\client.py", line 500, in read
return super(HTTPResponse, self).read(amt)
File "C:\Python34\lib\http\client.py", line 539, in readinto
n = self.fp.readinto(b)
File "C:\Python34\lib\socket.py", line 374, in readinto
return self._sock.recv_into(b)
File "C:\Python34\lib\ssl.py", line 751, in recv_into
return self.read(nbytes, buffer)
File "C:\Python34\lib\ssl.py", line 623, in read
v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\pip\basecommand.py", line 232, in main
status = self.run(options, args)
File "C:\Python34\lib\site-packages\pip\commands\install.py", line 339, in r
n
requirement_set.prepare_files(finder)
File "C:\Python34\lib\site-packages\pip\req\req_set.py", line 355, in prepar
_files
do_download, session=self.session,
File "C:\Python34\lib\site-packages\pip\download.py", line 782, in unpack_ur
session,
File "C:\Python34\lib\site-packages\pip\download.py", line 667, in unpack_ht
p_url
from_path, content_type = _download_http_url(link, session, temp_dir)
File "C:\Python34\lib\site-packages\pip\download.py", line 843, in _download
http_url
_download_url(resp, link, content_file)
File "C:\Python34\lib\site-packages\pip\download.py", line 615, in _download
url
for chunk in progress_indicator(resp_read(4096), 4096):
File "C:\Python34\lib\site-packages\pip\utils\ui.py", line 46, in iter
for x in it:
File "C:\Python34\lib\site-packages\pip\download.py", line 580, in resp_read
decode_content=False):
File "C:\Python34\lib\site-packages\pip\_vendor\requests\packages\urllib3\re
ponse.py", line 256, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Python34\lib\site-packages\pip\_vendor\requests\packages\urllib3\re
ponse.py", line 201, in read
raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip._vendor.requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPSConnec
ionPool(host='pypi.python.org', port=443): Read timed out.
=================================
第二部分
感觉可能是pip版本过低造成的，先升级下pip版本
pip install --upgrade pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |################################| 1.3MB 119kB/s
Installing collected packages: pip
Found existing installation: pip 6.0.8
Uninstalling pip-6.0.8:
Successfully uninstalled pip-6.0.8
Exception:
Traceback (most recent call last):
File "C:\Python34\lib\shutil.py", line 371, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] 拒绝访问。: 'C:\\DOCUME~1\\ADMINI~1\\LOCALS~1\\T
emp\\pip-d1yje83k-uninstall\\python34\\scripts\\pip.exe'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\pip\basecommand.py", line 232, in main
logger.debug('Exception information:', exc_info=True)
File "C:\Python34\lib\site-packages\pip\commands\install.py", line 347, in r
un
home=temp_target_dir,
File "C:\Python34\lib\site-packages\pip\req\req_set.py", line 560, in instal
l
# Then inside find_requirement existing_applicable -> False
File "C:\Python34\lib\site-packages\pip\req\req_install.py", line 680, in co
mmit_uninstall
)
File "C:\Python34\lib\site-packages\pip\req\req_uninstall.py", line 153, in
commit
self.file = pth_file
File "C:\Python34\lib\site-packages\pip\_vendor\retrying.py", line 49, in wr
apped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "C:\Python34\lib\site-packages\pip\_vendor\retrying.py", line 212, in c
all
raise attempt.get()
File "C:\Python34\lib\site-packages\pip\_vendor\retrying.py", line 247, in g
et
six.reraise(self.value[0], self.value[1], self.value[2])
File "C:\Python34\lib\site-packages\pip\_vendor\six.py", line 659, in rerais
e
_assertCountEqual = "assertItemsEqual"
File "C:\Python34\lib\site-packages\pip\_vendor\retrying.py", line 200, in c
all
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "C:\Python34\lib\site-packages\pip\utils\__init__.py", line 61, in rmtr
ee
SUPPORTED_EXTENSIONS += BZ2_EXTENSIONS
File "C:\Python34\lib\shutil.py", line 478, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Python34\lib\shutil.py", line 368, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python34\lib\shutil.py", line 368, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python34\lib\shutil.py", line 373, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Python34\lib\site-packages\pip\utils\__init__.py", line 73, in rmtr
ee_errorhandler
def import_or_raise(pkg_or_module_string, ExceptionType, *args, **kwargs):
PermissionError: [WinError 5] 拒绝访问。: 'C:\\DOCUME~1\\ADMINI~1\\LOCALS~1\\T
emp\\pip-d1yje83k-uninstall\\python34\\scripts\\pip.exe'
权限有问题，好吧，以管理员身份打开命令行
=============
运行pip --version ,这时已经是
pip 9.0.1 from C:\Python34\lib\site-packages (python 3.4)了
pip升级成功！！！
第三部分
在升级好了Pip之后继续安装scrapy
pip install scrapy
这次运行的时间稍微长了一些，但是仍然没有成功
copying src\twisted\words\xish\xpathparser.g -> build\lib.win32-3.4\twisted\
words\xish
running build_ext
building 'twisted.test.raiser' extension
error: Microsoft Visual C++ 10.0 is required (Unable to find vcvarsall.bat).
----------------------------------------
Command "C:\Python34\python.exe -u -c "import setuptools, tokenize;__file__='C:\
\DOCUME~1\\ADMINI~1\\LOCALS~1\\Temp\\pip-build-khxnalsf\\Twisted\\setup.py';f=ge
tattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.cl
ose();exec(compile(code, __file__, 'exec'))" install --record C:\DOCUME~1\ADMINI
~1\LOCALS~1\Temp\pip-e0vtso7q-record\install-record.txt --single-version-externa
lly-managed --compile" failed with error code 1 in C:\DOCUME~1\ADMINI~1\LOCALS~1
\Temp\pip-build-khxnalsf\Twisted\
意思就是twisted 没有安装
第四部分
通过查找资料，找到了如下链接
http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
找到对应的版本并下载
C:\Documents and Settings\Administrator>pip install "C:\Documents and Settings\A
dministrator\My Documents\下载\Twisted-17.5.0-cp34-cp34m-win32.whl"
Processing c:\documents and settings\administrator\my documents\下载\twisted-17.
5.0-cp34-cp34m-win32.whl
Requirement already satisfied: incremental>=16.10.1 in c:\python34\lib\site-pack
ages (from Twisted==17.5.0)
Requirement already satisfied: hyperlink>=17.1.1 in c:\python34\lib\site-package
s (from Twisted==17.5.0)
Requirement already satisfied: constantly>=15.1 in c:\python34\lib\site-packages
(from Twisted==17.5.0)
Requirement already satisfied: zope.interface>=4.0.2 in c:\python34\lib\site-pac
kages (from Twisted==17.5.0)
Requirement already satisfied: Automat>=0.3.0 in c:\python34\lib\site-packages (
from Twisted==17.5.0)
Requirement already satisfied: setuptools in c:\python34\lib\site-packages (from
zope.interface>=4.0.2->Twisted==17.5.0)
Requirement already satisfied: attrs in c:\python34\lib\site-packages (from Auto
mat>=0.3.0->Twisted==17.5.0)
Requirement already satisfied: six in c:\python34\lib\site-packages (from Automa
t>=0.3.0->Twisted==17.5.0)
Installing collected packages: Twisted
Successfully installed Twisted-17.5.0
可以看到，Twisted通过手动下载手动安装的方式成功安装了
第五部分
我们继续安装scrapy
pip install scrapy
Successfully installed asn1crypto-0.22.0 cffi-1.10.0 cryptography-2.0.3 cssselec
t-1.0.1 idna-2.6 lxml-3.8.0 parsel-1.2.0 pyOpenSSL-17.2.0 pyasn1-0.3.2 pyasn1-mo
dules-0.0.11 pycparser-2.18 queuelib-1.4.2 scrapy-1.4.0 service-identity-17.0.0
执行 scrapy命令
C:\Documents and Settings\Administrator>scrapy
Scrapy 1.4.0 - no active project
Usage:
scrapy <command> [options] [args]
Available commands:
bench Run quick benchmark test
fetch Fetch a URL using the Scrapy downloader
genspider Generate new spider using pre-defined templates
runspider Run a self-contained spider (without creating a project)
settings Get settings values
shell Interactive scraping console
startproject Create new project
version Print Scrapy version
view Open URL in browser, as seen by Scrapy
[ more ] More commands available when run from project directory
Use "scrapy <command> -h" to see more info about a command
C:\Documents and Settings\Administrator>
看来，历经千辛万苦，我们的Scrapy终于安装成功了。。。
==================================================
说明：
之前在另外一台win7-64位系统机器下安装scrapy并没有这么麻烦，
看来对32位的机器的支持越来越不好了。