实战爬取拷背漫画-Python

时间：2024-02-15 17:09:17

​

 一.抓包获取链接

以爬取《前科者》为例

获取搜索链接

https://api.copymanga.com/api/v3/search/comic?limit=5&q=前科者

获取漫画详细页面

https://api.copymanga.com/api/v3/comic/qiankezhe/group/default/chapters?limit=200

第一话的链接

https://api.copymanga.com/api/v3/comic/qiankezhe/chapter2/52f9d522-0d71-11eb-93ea-00163e0ca5bd

抓包过程中因为headers不同响应也不同，所以需要插入一句

headers = {"Uesr-Agent" : "Dart/2.10 (dart:io)", "region" : "1"}

 那么先引用requests模块，写出请求函数和创建目录函数

#请求
def request_get(url):
    try:
        headers = {"Uesr-Agent" : "Dart/2.10 (dart:io)", "region" : "1"}
        request_str = requests.get(url,headers=headers)
        request_str.encoding = 'utf-8-sig'
        return request_str
    except:
        print("访问失败，请检查网络")
        sys.exit()
#创建目录
def mkdir(path):
    folder = os.path.exists(path)
    if not folder:                   
        os.makedirs(path)            
    else:
　　　　print(path + "目录已存在")

 

二.处理搜索链接及其参数

利用抓包软件，获得搜索链接

https://api.copymanga.com/api/v3/search/comic?limit=5&q=前科者

其中参数limit为显示漫画搜索结果的数目，这个是模糊搜索，参数q就是搜索内容

既然是模糊搜索，那么当然要手动去app搜索一下，并几下想要的漫画排在第几

我们先把limit参数改为1，让链接只返回一部漫画，方便观察响应：

 1 {
 2     "code": 200,
 3     "message": "请求成功",
 4     "results": {
 5         "list": [
 6             {
 7                 "cover": "https://mirror2.mangafunc.fun/comic/qiankezhe/cover/3c1a6b4c-0d6b-11eb-b49c-00163e0ca5bd.jpg!kb_m_item",
 8                 "img_type": 2,
 9                 "author": [
10                     {
11                         "name": "月岛冬二",
12                         "alias": null,
13                         "path_word": "yuedaodonger"
14                     },
15                     {
16                         "name": "香川まさひと",
17                         "alias": null,
18                         "path_word": "xiangchuanirihm"
19                     }
20                 ],
21                 "name": "前科者",
22                 "alias": "前科者,前科者",
23                 "path_word": "qiankezhe",
24                 "popular": 13042
25             }
26         ],
27         "total": 1816,
28         "limit": 1,
29         "offset": 0
30     }
31 }

观察漫画详细页面的链接里面的qiankezhe与path_word对应，漫画名字对应name

那么我们根据这个模糊搜索，直接利用json模块获取这两个参数，并存放在name_words和names字典中，并对应上搜索结果的排列顺序，代码：

 1 try:
 2     name = input("请输入漫画名字：")
 3     th = int(input("请输入漫画搜索结果排序位置（数字）："))
 4 except:
 5     print("输入错误")
 6     sys.exit()
 7 
 8 search_url = "https://api.copymanga.com/api/v3/search/comic?limit=10&q={}".format(name)
 9 search_str = request_get(search_url)
10 name_str = json.loads(search_str.text).get("results").get("list")
11 name_words = {}
12 names = {}
13 num = 1
14 for i in name_str:
15     name_words.update({num : i.get("path_word")})
16     names.update({num : i.get("name")})
17     num += 1

 

三.处理漫画详细页面及其参数

https://api.copymanga.com/api/v3/comic/qiankezhe/group/default/chapters?limit=200

观察一下链接，limit明显是显示多少话，中间的qiankezhe在上面的name_words字典中

那么修改limit为1，来看一下响应（记得用json格式化，让响应好看一点）：

 1 {
 2     "code": 200,
 3     "message": "\u8bf7\u6c42\u6210\u529f",
 4     "results": {
 5         "list": [{
 6             "index": 0,
 7             "uuid": "52f9d522-0d71-11eb-93ea-00163e0ca5bd",
 8             "count": 28,
 9             "ordered": 10,
10             "size": 41,
11             "name": "第一话",
12             "comic_id": "3c192200-0d6b-11eb-b49c-00163e0ca5bd",
13             "comic_path_word": "qiankezhe",
14             "group_id": null,
15             "group_path_word": "default",
16             "type": 1,
17             "img_type": 2,
18             "datetime_created": "2020-10-14",
19             "prev": null,
20             "next": "cffb84a2-15d8-11eb-9e11-00163e0ca5bd"
21         }],
22         "total": 28,
23         "limit": 1,
24         "offset": 0
25     }
26 }

看到里面的uuid与第一话链接对应52f9d522-0d71-11eb-93ea-00163e0ca5bd

https://api.copymanga.com/api/v3/comic/qiankezhe/chapter2/52f9d522-0d71-11eb-93ea-00163e0ca5bd

那么我们创建这部漫话的目录，然后获取uuid和name（这个name就是第几话），再创建'第几话'目录，方便存放图片：

#创建漫画目录
name_path = path + r'\{}'.format(names[th])
mkdir(name_path)
#获取每一话的uuid
uuid_url = "https://api.copymanga.com/api/v3/comic/{}/group/default/chapters?limit=200".format(name_words[th])
uuid_str = request_get(uuid_url)
uuid_str = json.loads(uuid_str.text).get("results").get("list")
for i in uuid_str:
    chaper = i.get("name")
    uuid = i.get("uuid")
    chaper_path = name_path + r"\{}".format(chaper)
    mkdir(chaper_path) #创建'第几话'目录
    down(uuid) #加入用down函数下载图片（down函数暂未写出）

 

四.编写下载图片的down函数

先看一下第一话的链接

https://api.copymanga.com/api/v3/comic/qiankezhe/chapter2/52f9d522-0d71-11eb-93ea-00163e0ca5bd

qiankezhe和52f9d522-0d71-11eb-93ea-00163e0ca5bd已经在上面获取了，分别为name_words字典和字符串uuid

再看看响应(这个contents太长了，我就删除一些了）：

 1 {
 2     "code": 200,
 3     "message": "\u8bf7\u6c42\u6210\u529f",
 4     "results": {
 5         "show_app": false,
 6         "is_lock": false,
 7         "is_login": false,
 8         "is_mobile_bind": false,
 9         "is_vip": false,
10         "comic": {
11             "name": "\u524d\u79d1\u8005",
12             "uuid": "3c192200-0d6b-11eb-b49c-00163e0ca5bd",
13             "path_word": "qiankezhe",
14             "restrict": {
15                 "value": 0,
16                 "display": "\u4e00\u822c\u5411(\u514d\u8cbb)"
17             }
18         },
19         "chapter": {
20             "index": 0,
21             "uuid": "52f9d522-0d71-11eb-93ea-00163e0ca5bd",
22             "count": 28,
23             "ordered": 10,
24             "size": 41,
25             "name": "\u7b2c01\u8bdd",
26             "comic_id": "3c192200-0d6b-11eb-b49c-00163e0ca5bd",
27             "comic_path_word": "qiankezhe",
28             "group_id": null,
29             "group_path_word": "default",
30             "type": 1,
31             "img_type": 2,
32             "datetime_created": "2020-10-14",
33             "prev": null,
34             "next": "cffb84a2-15d8-11eb-9e11-00163e0ca5bd",
35             "contents": [{
36                 "uuid": "8aff0712-0d71-11eb-9be4-00163e0ca5bd",
37                 "url": "https://mirror2.mangafunc.fun/comic/qiankezhe/420b9/8afbbff8-0d71-11eb-9be4-00163e0ca5bd.jpg!kb_m_read_large"
38             }],
39             "words": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 30, 38, 39, 21, 31, 27, 32, 34, 23, 26, 28, 24, 40, 36, 20, 25, 33, 29, 22, 35, 37],
40             "is_long": false
41         }
42     }
43 }

 

可以看到contents里面的url就是每一张图片的链接，那么直接获取这个链接并下载到本地就行了

看代码：

 1 def down(uuid):
 2     get_pictrue_url = "https://api.copymanga.com/api/v3/comic/{}/chapter2/{}".format(name_words,uuid)
 3     picture_str = request_get(get_pictrue_url)
 4     picture_str = json.loads(picture_str.text).get("results").get("chapter").get("contents")
 5     num = 1
 6     for i in picture_str:
 7         picture = request_get(i.get("url"))
 8         with open(chaper_path + "\{}.jpg".format(num),"wb") as code:
 9             code.write(picture.content)
10         num = num + 1
 

结尾：

这样就可以了，我们引用了4个库：requests, sys, os, json 

整合全部代码：

 1 #!/usr/bin/python
 2 # -*- coding: UTF-8 -*-
 3 import requests
 4 import sys
 5 import os
 6 import json
 7 
 8 #############################配置区##############################
 9 #保存目录
10 path = r"D:\0file\Download"
11 #################################################################
12 
13 #################################################################
14 ############################代码区###############################
15 try:
16     name = input("请输入漫画名字：")
17     th = int(input("请输入漫画搜索结果排序位置（数字）："))
18 except:
19     print("输入错误")
20     sys.exit()
21 
22 #—————————————————————————————函数区————————————————————————————#
23 #创建目录
24 def mkdir(path):
25     folder = os.path.exists(path)
26     if not folder:                   
27         os.makedirs(path)            
28     else:
29         print (path + "目录已存在")
30 #请求
31 def request_get(url):
32     try:
33         headers = {"Uesr-Agent" : "Dart/2.10 (dart:io)", "region" : "1"}
34         request_str = requests.get(url,headers=headers)
35         request_str.encoding = 'utf-8-sig'
36         return request_str
37     except:
38         print("访问失败，请检查网络")
39         sys.exit()
40 #下再图片
41 def down(uuid):
42     get_pictrue_url = "https://api.copymanga.com/api/v3/comic/{}/chapter2/{}".format(name_words,uuid)
43     picture_str = request_get(get_pictrue_url)
44     picture_str = json.loads(picture_str.text).get("results").get("chapter").get("contents")
45     num = 1
46     for i in picture_str:
47         picture = request_get(i.get("url"))
48         with open(chaper_path + "\{}.jpg".format(num),"wb") as code:
49             code.write(picture.content)
50         num = num + 1
51 #——————————————————————————————————————————————————————————————#
52 
53 #获取搜索结果
54 search_url = "https://api.copymanga.com/api/v3/search/comic?limit=10&q={}".format(name)
55 search_str = request_get(search_url)
56 name_str = json.loads(search_str.text).get("results").get("list")
57 name_words = {}
58 names = {}
59 num = 1
60 for i in name_str:
61     name_words.update({num : i.get("path_word")})
62     names.update({num : i.get("name")})
63     num += 1
64 
65 #创建漫画目录
66 name_path = path + r'\{}'.format(names[th])
67 mkdir(name_path)
68 #获取每一话的uuid
69 uuid_url = "https://api.copymanga.com/api/v3/comic/{}/group/default/chapters?limit=200".format(name_words[th])
70 uuid_str = request_get(uuid_url)
71 uuid_str = json.loads(uuid_str.text).get("results").get("list")
72 for i in uuid_str:
73     chaper = i.get("name")
74     uuid = i.get("uuid")
75     chaper_path = name_path + r"\{}".format(chaper)
76     mkdir(chaper_path)
77     down(uuid)
78 #################################################################
79 #################################################################
80     

侵权删！请注意版权！仅供自己的编程学习与测试，不要将漫画进行传播，更不要牟利！尊重原作和内容提供商！

相关文章