python3利用beautiful soup获取网页文本及src链接和http链接

本人最近在学习python

https://github.com/Yixiaohan/show-me-the-code#rd

下面使用python3利用beautiful soup 这个强大的库解决第8题

代码如下

from urllib import request
from bs4 import BeautifulSoup

url = 'http://sports.163.com/special/unluckykaka/'
html = request.urlopen(url)

soup = BeautifulSoup(html.read(),"html.parser")
print(soup.body.text.encode('utf-8','ignore').decode('utf-8'))

若要获取html里面的src和url的信息，

利用beautiful soup 同样可以十分方便实现这个功能

from urllib import request
import re
from bs4 import BeautifulSoup

html = request.urlopen(r'file:///C:/Users/asus/Desktop/%E7%BD%91%E7%AB%99%E6%B5%8B%E8%AF%95.html').read()
soup = BeautifulSoup(html,"html.parser")

img_ = soup.find_all(name='img')
for each in img_:
    print(each.get('src'))
print('end')

href_ = soup.find_all(name='a')
for each in href_:
    if str(each.get('href'))[:4]=='http':
        print(each.get('href'))
print('end')