I am trying to learn how to automatically fetch urls from a page. In the following code I am trying to get the title of the webpage:
我正在尝试学习如何从一个页面自动获取url。在下面的代码中,我试图获得网页的标题:
import urllib.request
import re
url = "http://www.google.com"
regex = '<title>(,+?)</title>'
pattern = re.compile(regex)
with urllib.request.urlopen(url) as response:
html = response.read()
title = re.findall(pattern, html)
print(title)
And I get this unexpected error:
我得到了一个意想不到的错误:
Traceback (most recent call last):
File "path\to\file\Crawler.py", line 11, in <module>
title = re.findall(pattern, html)
File "C:\Python33\lib\re.py", line 201, in findall
return _compile(pattern, flags).findall(string)
TypeError: can't use a string pattern on a bytes-like object
What am I doing wrong?
我做错了什么?
Thanks!
谢谢!
1 个解决方案
#1
60
You want to convert html (a byte-like object) into a string using .decode
, e.g. html = response.read().decode('utf-8')
.
要将html(类似字节的对象)转换成字符串,使用.decode,例如html = response.read().decode('utf-8')。
See Convert bytes to a Python String
参见将字节转换为Python字符串。
#1
60
You want to convert html (a byte-like object) into a string using .decode
, e.g. html = response.read().decode('utf-8')
.
要将html(类似字节的对象)转换成字符串,使用.decode,例如html = response.read().decode('utf-8')。
See Convert bytes to a Python String
参见将字节转换为Python字符串。