最近右胳膊受伤,打了石膏在家休息。为了实现之前的想法,就用左手打字、写代码,查资料完成了这个资源小爬虫。网页爬虫,
最主要的是协议分析(必须要弄清楚自己的目的),另外就是要考虑对爬取的数据归类,存储。这是一个在线歌曲网站的爬虫,网站名
字就不说了,此贴目的是技术交流,请不用做其他用途!
相关技术点:http协议、js、AES、文件、文件夹操作、正则表达式、数据库技术、SQL
-------------------------------------------分割线 以下 为设计思路---------------------------------------------
思路:
1.打开fiddler,打开网站,在线试听
2.观察网页播放器的资源加载方式
3.搜索请求返回的报文,从中找到资源信息
4.分析资源信息的数据格式,找出规律
5.筛选出需要的数据,爬虫下载、存储
-------------------------------------------分割线 以下 遇到问题与解决思路,记录,以备后用---------------------------------------------
问题1:跟踪不到资源数据来源
在跟包过程中 发现了资源播放的加载路径,但搜索所求服务端返回的报文,均找不到对应的信息。无奈,只好把这网站的HTML、
JS 源码跟踪 ,静态分享 + 网页调试,最终得到答案:网页使用的资源播放列表是经过 AES 加密的,同时 密钥跟随页面一起发
送给了客户端(浏览器)
ps: eval("var playlist = " + GibberishAES.dec(pl, $.xxxx.xxx.aes))
问题2:在知道 密文、密钥的情况下,使用Python、Java、C#语言均无法解密
将从页面拿到的AES密文、密钥 通过在线AES解密网页(http://tool.chinaz.com/Tools/TextEncrypt.aspx)直接解开,考虑到
是AES,安装Crypto包,并进行加解密操作。实际情况是 使用 CBC模式,16位密钥长度,16位与密码一致的向量 无法实现与网页加解
密相同的结果(自己代码解不开播放列表密文)。后使用JAVA、C# 设置为相同模式,相同密钥,实现了C#、Java、Python之间的相互
加解密操作,但是与JS加解密(CryptoJS)仍不相同,到这就卡住了一天多。
无奈 硬头皮跟踪在线解密网站的JS,发现CryptoJS在加解密时候,会根据原始密钥 通过一定的算法(openssl kdf 算法?)得到新
的密钥、向量 并加盐处理! 很无奈 JS 语言能力不够,没办法看得懂那算法,也就没办法用其他语言实现了。在分析CryptoJS加密过程
的时候发现另外一个情况,就是相同的原文、密钥 每次加密的结果是不同的(有随机的加盐处理?),虽然每次密文都不同,但用相同的
密钥都能解得开,每次密文(Base64) 都会以U2FsdGVkX1开头 (很神奇,网上搜了下资料,应该是在密文中隐藏了密码盐 salt 随机数,
有朋友用C#实现类似的算法,见:http://www.cnblogs.com/stone_w/p/4229275.html)
ps:
相同明文、密钥 产生不同 密文(随机向量、盐?):
在线加解密:http://tool.chinaz.com/Tools/TextEncrypt.aspx
text: abcdefgcnblogs
key: 12345678
结果1:U2FsdGVkX19GtgRcaq8vxaWN8HjO+tJWZXmLBBbQv8c=
结果2:U2FsdGVkX18y6MsDEIxKqT0gS0AsoFsa9trUwnFzH5c=
结果3:U2FsdGVkX1/2rk/eD0MuFfJEe076aogxPnOacIDpqNs=
结果N:。。。
Java版本AES:
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/ /**
*
* @author Jacker
*/ import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import sun.misc.BASE64Decoder; public class Encryption
{
public static void main(String args[]) throws Exception {
System.out.println(encrypt());
System.out.println(desEncrypt());
} public static String encrypt() throws Exception {
try {
String data = "123456abcdefgyhn123456abcdefgyhn";
String key = "b6ce159334e155d8";
String iv = "b6ce159334e155d8"; Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
int blockSize = cipher.getBlockSize(); byte[] dataBytes = data.getBytes();
int plaintextLength = dataBytes.length;
if (plaintextLength % blockSize != 0) {
plaintextLength = plaintextLength + (blockSize - (plaintextLength % blockSize));
} byte[] plaintext = new byte[plaintextLength];
System.arraycopy(dataBytes, 0, plaintext, 0, dataBytes.length); SecretKeySpec keyspec = new SecretKeySpec(key.getBytes(), "AES");
IvParameterSpec ivspec = new IvParameterSpec(iv.getBytes()); cipher.init(Cipher.ENCRYPT_MODE, keyspec, ivspec);
byte[] encrypted = cipher.doFinal(plaintext); return new sun.misc.BASE64Encoder().encode(encrypted); } catch (Exception e) {
e.printStackTrace();
return null;
}
} public static String desEncrypt() throws Exception {
try
{
String data = "JooiPOsR21TbUksOLu21kZcR15RbEFtAhYn6VKdRoJw=";
String key = "b6ce159334e155d8";
String iv = "b6ce159334e155d8"; byte[] encrypted1 = new BASE64Decoder().decodeBuffer(data); Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
SecretKeySpec keyspec = new SecretKeySpec(key.getBytes(), "AES");
IvParameterSpec ivspec = new IvParameterSpec(iv.getBytes()); cipher.init(Cipher.DECRYPT_MODE, keyspec, ivspec); byte[] original = cipher.doFinal(encrypted1);
String originalString = new String(original);
return originalString;
}
catch (Exception e) {
e.printStackTrace();
return null;
}
}
}
C#版本AES:
using System;
using System.Text;
using System.Security.Cryptography;
using System.IO; namespace ConsoleAppAESEncrypt
{
/// <summary>
/// DES加密解密
/// </summary>
public class DES
{
/// <summary>
/// 获取密钥
/// </summary>
private static string Key
{
get { return @"P@+#wG+Z"; }
} /// <summary>
/// 获取向量
/// </summary>
private static string IV
{
get { return @"L%n67}G\Mk@k%:~Y"; }
} /// <summary>
/// DES加密
/// </summary>
/// <param name="plainStr">明文字符串</param>
/// <returns>密文</returns>
public static string DESEncrypt(string plainStr)
{
byte[] bKey = Encoding.UTF8.GetBytes(Key);
byte[] bIV = Encoding.UTF8.GetBytes(IV);
byte[] byteArray = Encoding.UTF8.GetBytes(plainStr); string encrypt = null;
DESCryptoServiceProvider des = new DESCryptoServiceProvider();
try
{
using (MemoryStream mStream = new MemoryStream())
{
using (CryptoStream cStream = new CryptoStream(mStream, des.CreateEncryptor(bKey, bIV), CryptoStreamMode.Write))
{
cStream.Write(byteArray, 0, byteArray.Length);
cStream.FlushFinalBlock();
encrypt = Convert.ToBase64String(mStream.ToArray());
}
}
}
catch { }
des.Clear(); return encrypt;
} /// <summary>
/// DES解密
/// </summary>
/// <param name="encryptStr">密文字符串</param>
/// <returns>明文</returns>
public static string DESDecrypt(string encryptStr)
{
byte[] bKey = Encoding.UTF8.GetBytes(Key);
byte[] bIV = Encoding.UTF8.GetBytes(IV);
byte[] byteArray = Convert.FromBase64String(encryptStr); string decrypt = null;
DESCryptoServiceProvider des = new DESCryptoServiceProvider();
try
{
using (MemoryStream mStream = new MemoryStream())
{
using (CryptoStream cStream = new CryptoStream(mStream, des.CreateDecryptor(bKey, bIV), CryptoStreamMode.Write))
{
cStream.Write(byteArray, 0, byteArray.Length);
cStream.FlushFinalBlock();
decrypt = Encoding.UTF8.GetString(mStream.ToArray());
}
}
}
catch { }
des.Clear(); return decrypt;
}
} /// <summary>
/// AES加密解密
/// </summary>
public class AES
{
/// <summary>
/// 获取密钥
/// </summary>
private static string Key
{
// get { return @")O[NB]6,YF}+efcaj{+oESb9d8>Z'e9M"; }
get { return @"b6ce159334e155d8"; }
} /// <summary>
/// 获取向量
/// </summary>
private static string IV
{
// get { return @"L+\~f4,Ir)b$=pkf"; }
get { return @"b6ce159334e155d8"; }
} /// <summary>
/// AES加密
/// </summary>
/// <param name="plainStr">明文字符串</param>
/// <returns>密文</returns>
public static string AESEncrypt(string plainStr)
{
byte[] bKey = Encoding.UTF8.GetBytes(Key);
byte[] bIV = Encoding.UTF8.GetBytes(IV);
byte[] byteArray = Encoding.UTF8.GetBytes(plainStr); string encrypt = null;
Rijndael aes = Rijndael.Create();
aes.Mode = CipherMode.CBC;
aes.Padding = PaddingMode.Zeros;
try
{
using (MemoryStream mStream = new MemoryStream())
{
using (CryptoStream cStream = new CryptoStream(mStream, aes.CreateEncryptor(bKey, bIV), CryptoStreamMode.Write))
{
cStream.Write(byteArray, 0, byteArray.Length);
cStream.FlushFinalBlock();
encrypt = Convert.ToBase64String(mStream.ToArray());
}
}
}
catch { }
aes.Clear(); return encrypt;
} /// <summary>
/// AES加密
/// </summary>
/// <param name="plainStr">明文字符串</param>
/// <param name="returnNull">加密失败时是否返回 null,false 返回 String.Empty</param>
/// <returns>密文</returns>
public static string AESEncrypt(string plainStr, bool returnNull)
{
string encrypt = AESEncrypt(plainStr);
return returnNull ? encrypt : (encrypt == null ? String.Empty : encrypt);
} /// <summary>
/// AES解密
/// </summary>
/// <param name="encryptStr">密文字符串</param>
/// <returns>明文</returns>
public static string AESDecrypt(string encryptStr)
{
byte[] bKey = Encoding.UTF8.GetBytes(Key);
byte[] bIV = Encoding.UTF8.GetBytes(IV);
byte[] byteArray = Convert.FromBase64String(encryptStr); string decrypt = null;
Rijndael aes = Rijndael.Create();
aes.Mode = CipherMode.CBC;
aes.Padding = PaddingMode.Zeros;
try
{
using (MemoryStream mStream = new MemoryStream())
{
using (CryptoStream cStream = new CryptoStream(mStream, aes.CreateDecryptor(bKey, bIV), CryptoStreamMode.Write))
{
cStream.Write(byteArray, 0, byteArray.Length);
cStream.FlushFinalBlock();
decrypt = Encoding.UTF8.GetString(mStream.ToArray());
}
}
}
catch { }
aes.Clear(); return decrypt;
} /// <summary>
/// AES解密
/// </summary>
/// <param name="encryptStr">密文字符串</param>
/// <param name="returnNull">解密失败时是否返回 null,false 返回 String.Empty</param>
/// <returns>明文</returns>
public static string AESDecrypt(string encryptStr, bool returnNull)
{
string decrypt = AESDecrypt(encryptStr);
return returnNull ? decrypt : (decrypt == null ? String.Empty : decrypt);
}
}
}
Python版本AES:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from Crypto.Cipher import AES
import base64
# PADDING = '\0'
PADDING = ' '
# pad_it = lambda s: s + (16 - len(s) % 16) * PADDING
# pad_it = lambda s: s# + (32 - len(s) % 32) * PADDING def pad_it(text):
length = 16
# count = text.count('')
count = len(text) # .count('')
padcount = (count % length)
if padcount == 0:
return text add = length - padcount
text = text + (PADDING * add)
return text
pass key = 'b6ce159334e155d8'
iv = 'b6ce159334e155d8'
source = '123456abcdefgyhn123456abcdefgyhn' print 'source len:', len(source), source generator = AES.new(key, AES.MODE_CBC, iv)
source = pad_it(source) crypt = generator.encrypt(source)
cryptedStr = base64.b64encode(crypt)
print cryptedStr
generator = AES.new(key, AES.MODE_CBC, iv)
recovery = generator.decrypt(crypt) print len(recovery), len(recovery.rstrip(PADDING)), recovery.rstrip(PADDING)
问题3:如何在pytyon内调用JS实现解密
卡在AES解密上无解一天后,在群里问问其他朋友,在此感谢 @寻找和谐 给的思路,用虚机货其他方式加载并执行JS,实现
解密,真是一语惊醒梦中人。网上查了资料,在Python中执行JS至少有3种方案,考虑到我是系统是windows,就用了比较简单的
方式 调用控件MSScriptControl.ScriptControl
(需要安装 pywin32-214.win32-py2.7)资料参见:
http://www.360doc.com/content/13/0318/15/7492958_272244611.shtml
http://blog.chinaunix.net/uid-9407860-id-2423996.html
-------------------------------------------分割线 以下 为本次爬虫相关代码---------------------------------------------
备注:爬虫除了获取点点音频文件外,还将解析的数据存放到了sqlite中(建表脚本在py代码中)
代码结构:
相关代码:
pywinjsaes.py
#! /usr/bin/env python
# coding=utf-8 import win32com.server.util, win32com.client # 以下代码解决输出乱码问题
import sys
# print sys.getdefaultencoding()
reload(sys)
sys.setdefaultencoding('utf8')
# print sys.getdefaultencoding() class __PyWinJsAes: def __init__(self): js = win32com.client.Dispatch('MSScriptControl.ScriptControl')
js.Language = 'JavaScript'
js.AllowUI = False
js.AddCode(self.__readJsFile("jsfiles/aes.js"))
js.AddCode(self.__readJsFile("jsfiles/aesutil.js"))
self.jsengine = js def __readJsFile(self, filename): fp = file(filename, 'r')
lines = ''
for line in fp:
lines += line
return lines def __driveJsCode(self, func, paras): if paras:
return self.jsengine.Run(func, paras[0], paras[1])
else:
return self.jsengine.Run(func) def encrypt(self, text, key):
return self.__driveJsCode("DoAesEncrypt", [text, key]) def decrypt(self, text, key):
# print text,key
return self.__driveJsCode("DoAesDecrypt", [text, key]) JsAes = __PyWinJsAes() if __name__ == '__main__': p = JsAes.decrypt("U2FsdGVkX19FDZhhIeMCH9SHfLg8B34NUbWxnuRFtc++fkhyKov9urtLuG7qatqm TP2/LEy+g35Jarbm5KoGCg==",
"456")
print '*' * 20
print p # js.run
luomusic.py
#! /usr/bin/env python
# coding=utf-8 import requests
import json
import re
import struct
import base64
import sqlite3
import os
import urllib
import urllib2
import time
import random from pywinjsaes import JsAes # 以下代码解决输出乱码问题
import sys
# print sys.getdefaultencoding()
reload(sys)
sys.setdefaultencoding('utf8')
# print sys.getdefaultencoding() re_win_filename_pattern = '\\\|/|:|\*|\?|\"|<|>|\|'
re_win_filename = re.compile(re_win_filename_pattern) class LuoMusic: def __init__(self):
self.RootDir = "LuooMusic/"
self.AudioDir = self.RootDir + "AudioFile/"
self.ReqHeader = self.__initReqHeader()
self.ReqRootUrl = "http://www.luoo.net/music/" self.dbcx = sqlite3.connect(self.RootDir + "luomusic.sqlite")
pass def __initReqHeader(self): # Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0'
}
return headers def getAlbumInfoByIndex(self, index): errRet = (None, None, None) # 发起请求
try:
pageurl = self.ReqRootUrl + str(index)
print '抓取:', pageurl,
r = requests.get(pageurl, headers=self.ReqHeader)
r.encoding = 'utf-8'
# print r.encoding, r.text
luohtml = r.text print '状态:', r.status_code
except Exception, e:
print '发生异常:[%s] 继续处理下一个...' % e
return errRet # 从HTML中正则解析专辑信息 # 获取 标题 <span class="vol-title">关于失眠和夜晚的世界</span>
titlepattern = '"vol-title">(.*)</span>'
titlelist = re.compile(titlepattern).findall(luohtml)
if len(titlelist) == 0:
print '解析页面失败:标题为空 ,继续处理下一个...'
return errRet
title = titlelist[0] # 获取 描述 "vol-desc">本期音乐为*后摇音乐专题。<br></div><div class="clearfix vol-meta">
descpattern = '<div class="vol-desc">([\s\S]*)</div>\s*<div class="clearfix vol-meta">'
desclist = re.compile(descpattern).findall(luohtml)
if len(desclist) == 0:
print '解析页面失败:描述为空 ,继续处理下一个...'
return errRet
desc = desclist[0] re_br = re.compile('<br\s*?/?>') # 处理换行
desc = re_br.sub('\n', desc)
# re_br=re.compile('<br\s*?/?>|\s')#去除所有换行空格<br/>
# desc=re_br.sub('',desc)
# print "desc:", len(desc), desc # 获取加密 歌单 var pl = "U2FsdGVkX1/";
playListPattern = 'pl\s*=\s*\"(\S*)";'
playListStrs = re.compile(playListPattern).findall(luohtml)
if len(playListStrs) == 0:
print '解析页面失败:歌单为空 ,继续处理下一个...'
return errRet
playList = playListStrs[0] # 获取加密 密钥 "aes":"b6ce159334e155d8"}}']
aeskeypattern = '"aes":"(.*)"\}\}'
aeskeylist = re.compile(aeskeypattern).findall(luohtml)
if len(aeskeylist) == 0:
print '解析页面失败:密钥为空 ,继续处理下一个...'
return errRet
aeskey = aeskeylist[0] # 解密并返回结果
playList = JsAes.decrypt(playList, aeskey)
return title, desc, playList def saveAlbumInfo(self, ablumindex, title, desc , playList):
"""
CREATE TABLE "AlbumInfo" ("albumindex" INTEGER PRIMARY KEY NOT NULL DEFAULT (null) ,"title" TEXT,"desc" TEXT,"playlist" TEXT, "dir" TEXT)
CREATE TABLE "MusicInfo" ("albumindex" INTEGER NOT NULL , "musicindex" INTEGER NOT NULL ,"filename" TEXT,"dir" TEXT, "id" TEXT, "title" TEXT, "artist" TEXT, "album" TEXT, "mp3" TEXT, "poster" TEXT, "poster_small" TEXT, "is_fav" INTEGER, PRIMARY KEY ("albumindex", "musicindex")) """ # 创建专辑文件夹,并存放专辑信息
foldername = '%s%s-%s' % (self.AudioDir, ablumindex, title)
print '创建文件夹:%s ' % foldername,
if not os.path.exists(foldername) or not os.path.isdir(foldername):
os.mkdir(foldername)
print '成功'
else:
print '已存在' # 将专辑信息记录一份为文本 存放在专辑目录下
abluminftxt = foldername + '/0-%s.txt' % title
print '创建文件 :%s ' % abluminftxt,
if os.path.exists(abluminftxt) and os.path.isfile(abluminftxt):
print '已存在'
else:
file_object = open(abluminftxt, 'w')
try:
file_object.writelines('%s<$>%s<$>%s<$>%s' % (ablumindex, title, desc , playList))
except Exception, e:
print '发生异常:', e
finally:
file_object.close()
print '成功 ' # 循环处理歌曲信息 保存文件
songinsvalues = []
songdelvalues = []
songlistjson = json.loads(playList)
songindex = 0
sleepsecond = 2 print '开始循环下载,一共%s个文件...' % len(songlistjson)
for s in songlistjson:
songindex = songindex + 1 # 计算随机休眠时间
if songindex % 5 == 0:
sleepsecond = 10
else:
sleepsecond = random.randint(2, 5)
# 写死休眠时间
sleepsecond = 1 # 处理下载、存储名称
url = s["mp3"]
title = re_win_filename.sub('', s["title"])
filename = title + os.path.splitext(url)[-1]
localfilepath = foldername + '/' + filename
print "%s 下载=>[%s]:休眠:[%s]s %s => %s " % \
(time.strftime("%Y-%m-%d %H:%M:%S"), songindex, sleepsecond, url, localfilepath), # 下载文件
try:
# # urllib方式下载
# urllib.urlretrieve(url, localfilepath) # # urllib2方式下载
# f = urllib2.urlopen(url)
# data = f.read()
# with open(localfilepath, "wb") as code:
# code.write(data) # requests 方式下载
if os.path.exists(localfilepath) and os.path.isfile(localfilepath):
print '已存在'
else:
time.sleep(sleepsecond)
r = requests.get(url, timeout=15, headers=self.ReqHeader)
with open(localfilepath, "wb") as code:
code.write(r.content)
print '成功' except Exception , e:
print '异常'
print 'errmsg:[%s] 继续下一个...' % e
filename = ''
sleepsecond = 10 # 将歌曲信息存入列表 存储数据时使用
songdelvalues.append((ablumindex, songindex))
songinsvalues.append((ablumindex, songindex, filename, foldername, s["id"], title\
, s["artist"], s["album"], s["mp3"], s["poster"], s["poster_small"], s["is_fav"])) # 记录信息到数据库
cur = self.dbcx.cursor() # 专辑信息
delSql = "delete from AlbumInfo where albumindex=%s;" % ablumindex
cur.execute(delSql) insSql = 'insert into AlbumInfo values(?,?,?,?,?);'
cur.execute(insSql, (ablumindex, title, desc , playList, foldername)) print '提交专辑信息到数据库...'
self.dbcx.commit() # 歌曲信息
delSql = "delete from MusicInfo where albumindex=? and musicindex=?;"
cur.executemany(delSql, songdelvalues) insSql = 'insert into MusicInfo values(?,?,?,?,?,?,?,?,?,?,?,?);'
cur.executemany(insSql, songinsvalues) print '提交歌曲列表信息到数据库...'
self.dbcx.commit() # 关闭游标
cur.close()
pass if __name__ == "__main__": luo = LuoMusic() for ablumindex in range(500, 720): print '*' * 20, '开始处理第%s个专辑' % ablumindex, '*' * 20 title, desc, playList = luo.getAlbumInfoByIndex(ablumindex)
if title is None:
time.sleep(60 * 2)
continue title = re_win_filename.sub('', title)
print '解析成功,专辑名称为:', title print '开始保存专辑信息...'
luo.saveAlbumInfo(ablumindex, title, desc , playList) print '*' * 20, '处理完成.', '*' * 20
aes.js
/*
CryptoJS v3.0.2
code.google.com/p/crypto-js
(c) 2009-2012 by Jeff Mott. All rights reserved.
code.google.com/p/crypto-js/wiki/License
*/
var CryptoJS=CryptoJS||function(p,h){var i={},l=i.lib={},r=l.Base=function(){function a(){}return{extend:function(e){a.prototype=this;var c=new a;e&&c.mixIn(e);c.$super=this;return c},create:function(){var a=this.extend();a.init.apply(a,arguments);return a},init:function(){},mixIn:function(a){for(var c in a)a.hasOwnProperty(c)&&(this[c]=a[c]);a.hasOwnProperty("toString")&&(this.toString=a.toString)},clone:function(){return this.$super.extend(this)}}}(),o=l.WordArray=r.extend({init:function(a,e){a=
this.words=a||[];this.sigBytes=e!=h?e:4*a.length},toString:function(a){return(a||s).stringify(this)},concat:function(a){var e=this.words,c=a.words,b=this.sigBytes,a=a.sigBytes;this.clamp();if(b%4)for(var d=0;d<a;d++)e[b+d>>>2]|=(c[d>>>2]>>>24-8*(d%4)&255)<<24-8*((b+d)%4);else if(65535<c.length)for(d=0;d<a;d+=4)e[b+d>>>2]=c[d>>>2];else e.push.apply(e,c);this.sigBytes+=a;return this},clamp:function(){var a=this.words,e=this.sigBytes;a[e>>>2]&=4294967295<<32-8*(e%4);a.length=p.ceil(e/4)},clone:function(){var a=
r.clone.call(this);a.words=this.words.slice(0);return a},random:function(a){for(var e=[],c=0;c<a;c+=4)e.push(4294967296*p.random()|0);return o.create(e,a)}}),m=i.enc={},s=m.Hex={stringify:function(a){for(var e=a.words,a=a.sigBytes,c=[],b=0;b<a;b++){var d=e[b>>>2]>>>24-8*(b%4)&255;c.push((d>>>4).toString(16));c.push((d&15).toString(16))}return c.join("")},parse:function(a){for(var e=a.length,c=[],b=0;b<e;b+=2)c[b>>>3]|=parseInt(a.substr(b,2),16)<<24-4*(b%8);return o.create(c,e/2)}},n=m.Latin1={stringify:function(a){for(var e=
a.words,a=a.sigBytes,c=[],b=0;b<a;b++)c.push(String.fromCharCode(e[b>>>2]>>>24-8*(b%4)&255));return c.join("")},parse:function(a){for(var e=a.length,c=[],b=0;b<e;b++)c[b>>>2]|=(a.charCodeAt(b)&255)<<24-8*(b%4);return o.create(c,e)}},k=m.Utf8={stringify:function(a){try{return decodeURIComponent(escape(n.stringify(a)))}catch(e){throw Error("Malformed UTF-8 data");}},parse:function(a){return n.parse(unescape(encodeURIComponent(a)))}},f=l.BufferedBlockAlgorithm=r.extend({reset:function(){this._data=o.create();
this._nDataBytes=0},_append:function(a){"string"==typeof a&&(a=k.parse(a));this._data.concat(a);this._nDataBytes+=a.sigBytes},_process:function(a){var e=this._data,c=e.words,b=e.sigBytes,d=this.blockSize,q=b/(4*d),q=a?p.ceil(q):p.max((q|0)-this._minBufferSize,0),a=q*d,b=p.min(4*a,b);if(a){for(var j=0;j<a;j+=d)this._doProcessBlock(c,j);j=c.splice(0,a);e.sigBytes-=b}return o.create(j,b)},clone:function(){var a=r.clone.call(this);a._data=this._data.clone();return a},_minBufferSize:0});l.Hasher=f.extend({init:function(){this.reset()},
reset:function(){f.reset.call(this);this._doReset()},update:function(a){this._append(a);this._process();return this},finalize:function(a){a&&this._append(a);this._doFinalize();return this._hash},clone:function(){var a=f.clone.call(this);a._hash=this._hash.clone();return a},blockSize:16,_createHelper:function(a){return function(e,c){return a.create(c).finalize(e)}},_createHmacHelper:function(a){return function(e,c){return g.HMAC.create(a,c).finalize(e)}}});var g=i.algo={};return i}(Math);
(function(){var p=CryptoJS,h=p.lib.WordArray;p.enc.Base64={stringify:function(i){var l=i.words,h=i.sigBytes,o=this._map;i.clamp();for(var i=[],m=0;m<h;m+=3)for(var s=(l[m>>>2]>>>24-8*(m%4)&255)<<16|(l[m+1>>>2]>>>24-8*((m+1)%4)&255)<<8|l[m+2>>>2]>>>24-8*((m+2)%4)&255,n=0;4>n&&m+0.75*n<h;n++)i.push(o.charAt(s>>>6*(3-n)&63));if(l=o.charAt(64))for(;i.length%4;)i.push(l);return i.join("")},parse:function(i){var i=i.replace(/\s/g,""),l=i.length,r=this._map,o=r.charAt(64);o&&(o=i.indexOf(o),-1!=o&&(l=o));
for(var o=[],m=0,s=0;s<l;s++)if(s%4){var n=r.indexOf(i.charAt(s-1))<<2*(s%4),k=r.indexOf(i.charAt(s))>>>6-2*(s%4);o[m>>>2]|=(n|k)<<24-8*(m%4);m++}return h.create(o,m)},_map:"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="}})();
(function(p){function h(f,g,a,e,c,b,d){f=f+(g&a|~g&e)+c+d;return(f<<b|f>>>32-b)+g}function i(f,g,a,e,c,b,d){f=f+(g&e|a&~e)+c+d;return(f<<b|f>>>32-b)+g}function l(f,g,a,e,c,b,d){f=f+(g^a^e)+c+d;return(f<<b|f>>>32-b)+g}function r(f,g,a,e,c,b,d){f=f+(a^(g|~e))+c+d;return(f<<b|f>>>32-b)+g}var o=CryptoJS,m=o.lib,s=m.WordArray,m=m.Hasher,n=o.algo,k=[];(function(){for(var f=0;64>f;f++)k[f]=4294967296*p.abs(p.sin(f+1))|0})();n=n.MD5=m.extend({_doReset:function(){this._hash=s.create([1732584193,4023233417,
2562383102,271733878])},_doProcessBlock:function(f,g){for(var a=0;16>a;a++){var e=g+a,c=f[e];f[e]=(c<<8|c>>>24)&16711935|(c<<24|c>>>8)&4278255360}for(var e=this._hash.words,c=e[0],b=e[1],d=e[2],q=e[3],a=0;64>a;a+=4)16>a?(c=h(c,b,d,q,f[g+a],7,k[a]),q=h(q,c,b,d,f[g+a+1],12,k[a+1]),d=h(d,q,c,b,f[g+a+2],17,k[a+2]),b=h(b,d,q,c,f[g+a+3],22,k[a+3])):32>a?(c=i(c,b,d,q,f[g+(a+1)%16],5,k[a]),q=i(q,c,b,d,f[g+(a+6)%16],9,k[a+1]),d=i(d,q,c,b,f[g+(a+11)%16],14,k[a+2]),b=i(b,d,q,c,f[g+a%16],20,k[a+3])):48>a?(c=
l(c,b,d,q,f[g+(3*a+5)%16],4,k[a]),q=l(q,c,b,d,f[g+(3*a+8)%16],11,k[a+1]),d=l(d,q,c,b,f[g+(3*a+11)%16],16,k[a+2]),b=l(b,d,q,c,f[g+(3*a+14)%16],23,k[a+3])):(c=r(c,b,d,q,f[g+3*a%16],6,k[a]),q=r(q,c,b,d,f[g+(3*a+7)%16],10,k[a+1]),d=r(d,q,c,b,f[g+(3*a+14)%16],15,k[a+2]),b=r(b,d,q,c,f[g+(3*a+5)%16],21,k[a+3]));e[0]=e[0]+c|0;e[1]=e[1]+b|0;e[2]=e[2]+d|0;e[3]=e[3]+q|0},_doFinalize:function(){var f=this._data,g=f.words,a=8*this._nDataBytes,e=8*f.sigBytes;g[e>>>5]|=128<<24-e%32;g[(e+64>>>9<<4)+14]=(a<<8|a>>>
24)&16711935|(a<<24|a>>>8)&4278255360;f.sigBytes=4*(g.length+1);this._process();f=this._hash.words;for(g=0;4>g;g++)a=f[g],f[g]=(a<<8|a>>>24)&16711935|(a<<24|a>>>8)&4278255360}});o.MD5=m._createHelper(n);o.HmacMD5=m._createHmacHelper(n)})(Math);
(function(){var p=CryptoJS,h=p.lib,i=h.Base,l=h.WordArray,h=p.algo,r=h.EvpKDF=i.extend({cfg:i.extend({keySize:4,hasher:h.MD5,iterations:1}),init:function(i){this.cfg=this.cfg.extend(i)},compute:function(i,m){for(var h=this.cfg,n=h.hasher.create(),k=l.create(),f=k.words,g=h.keySize,h=h.iterations;f.length<g;){a&&n.update(a);var a=n.update(i).finalize(m);n.reset();for(var e=1;e<h;e++)a=n.finalize(a),n.reset();k.concat(a)}k.sigBytes=4*g;return k}});p.EvpKDF=function(i,l,h){return r.create(h).compute(i,
l)}})();
CryptoJS.lib.Cipher||function(p){var h=CryptoJS,i=h.lib,l=i.Base,r=i.WordArray,o=i.BufferedBlockAlgorithm,m=h.enc.Base64,s=h.algo.EvpKDF,n=i.Cipher=o.extend({cfg:l.extend(),createEncryptor:function(b,d){return this.create(this._ENC_XFORM_MODE,b,d)},createDecryptor:function(b,d){return this.create(this._DEC_XFORM_MODE,b,d)},init:function(b,d,a){this.cfg=this.cfg.extend(a);this._xformMode=b;this._key=d;this.reset()},reset:function(){o.reset.call(this);this._doReset()},process:function(b){this._append(b);return this._process()},
finalize:function(b){b&&this._append(b);return this._doFinalize()},keySize:4,ivSize:4,_ENC_XFORM_MODE:1,_DEC_XFORM_MODE:2,_createHelper:function(){return function(b){return{encrypt:function(a,q,j){return("string"==typeof q?c:e).encrypt(b,a,q,j)},decrypt:function(a,q,j){return("string"==typeof q?c:e).decrypt(b,a,q,j)}}}}()});i.StreamCipher=n.extend({_doFinalize:function(){return this._process(!0)},blockSize:1});var k=h.mode={},f=i.BlockCipherMode=l.extend({createEncryptor:function(b,a){return this.Encryptor.create(b,
a)},createDecryptor:function(b,a){return this.Decryptor.create(b,a)},init:function(b,a){this._cipher=b;this._iv=a}}),k=k.CBC=function(){function b(b,a,d){var c=this._iv;c?this._iv=p:c=this._prevBlock;for(var e=0;e<d;e++)b[a+e]^=c[e]}var a=f.extend();a.Encryptor=a.extend({processBlock:function(a,d){var c=this._cipher,e=c.blockSize;b.call(this,a,d,e);c.encryptBlock(a,d);this._prevBlock=a.slice(d,d+e)}});a.Decryptor=a.extend({processBlock:function(a,d){var c=this._cipher,e=c.blockSize,f=a.slice(d,d+
e);c.decryptBlock(a,d);b.call(this,a,d,e);this._prevBlock=f}});return a}(),g=(h.pad={}).Pkcs7={pad:function(b,a){for(var c=4*a,c=c-b.sigBytes%c,e=c<<24|c<<16|c<<8|c,f=[],g=0;g<c;g+=4)f.push(e);c=r.create(f,c);b.concat(c)},unpad:function(b){b.sigBytes-=b.words[b.sigBytes-1>>>2]&255}};i.BlockCipher=n.extend({cfg:n.cfg.extend({mode:k,padding:g}),reset:function(){n.reset.call(this);var b=this.cfg,a=b.iv,b=b.mode;if(this._xformMode==this._ENC_XFORM_MODE)var c=b.createEncryptor;else c=b.createDecryptor,
this._minBufferSize=1;this._mode=c.call(b,this,a&&a.words)},_doProcessBlock:function(b,a){this._mode.processBlock(b,a)},_doFinalize:function(){var b=this.cfg.padding;if(this._xformMode==this._ENC_XFORM_MODE){b.pad(this._data,this.blockSize);var a=this._process(!0)}else a=this._process(!0),b.unpad(a);return a},blockSize:4});var a=i.CipherParams=l.extend({init:function(a){this.mixIn(a)},toString:function(a){return(a||this.formatter).stringify(this)}}),k=(h.format={}).OpenSSL={stringify:function(a){var d=
a.ciphertext,a=a.salt,d=(a?r.create([1398893684,1701076831]).concat(a).concat(d):d).toString(m);return d=d.replace(/(.{64})/g,"$1\n")},parse:function(b){var b=m.parse(b),d=b.words;if(1398893684==d[0]&&1701076831==d[1]){var c=r.create(d.slice(2,4));d.splice(0,4);b.sigBytes-=16}return a.create({ciphertext:b,salt:c})}},e=i.SerializableCipher=l.extend({cfg:l.extend({format:k}),encrypt:function(b,d,c,e){var e=this.cfg.extend(e),f=b.createEncryptor(c,e),d=f.finalize(d),f=f.cfg;return a.create({ciphertext:d,
key:c,iv:f.iv,algorithm:b,mode:f.mode,padding:f.padding,blockSize:b.blockSize,formatter:e.format})},decrypt:function(a,c,e,f){f=this.cfg.extend(f);c=this._parse(c,f.format);return a.createDecryptor(e,f).finalize(c.ciphertext)},_parse:function(a,c){return"string"==typeof a?c.parse(a):a}}),h=(h.kdf={}).OpenSSL={compute:function(b,c,e,f){f||(f=r.random(8));b=s.create({keySize:c+e}).compute(b,f);e=r.create(b.words.slice(c),4*e);b.sigBytes=4*c;return a.create({key:b,iv:e,salt:f})}},c=i.PasswordBasedCipher=
e.extend({cfg:e.cfg.extend({kdf:h}),encrypt:function(a,c,f,j){j=this.cfg.extend(j);f=j.kdf.compute(f,a.keySize,a.ivSize);j.iv=f.iv;a=e.encrypt.call(this,a,c,f.key,j);a.mixIn(f);return a},decrypt:function(a,c,f,j){j=this.cfg.extend(j);c=this._parse(c,j.format);f=j.kdf.compute(f,a.keySize,a.ivSize,c.salt);j.iv=f.iv;return e.decrypt.call(this,a,c,f.key,j)}})}();
(function(){var p=CryptoJS,h=p.lib.BlockCipher,i=p.algo,l=[],r=[],o=[],m=[],s=[],n=[],k=[],f=[],g=[],a=[];(function(){for(var c=[],b=0;256>b;b++)c[b]=128>b?b<<1:b<<1^283;for(var d=0,e=0,b=0;256>b;b++){var j=e^e<<1^e<<2^e<<3^e<<4,j=j>>>8^j&255^99;l[d]=j;r[j]=d;var i=c[d],h=c[i],p=c[h],t=257*c[j]^16843008*j;o[d]=t<<24|t>>>8;m[d]=t<<16|t>>>16;s[d]=t<<8|t>>>24;n[d]=t;t=16843009*p^65537*h^257*i^16843008*d;k[j]=t<<24|t>>>8;f[j]=t<<16|t>>>16;g[j]=t<<8|t>>>24;a[j]=t;d?(d=i^c[c[c[p^i]]],e^=c[c[e]]):d=e=1}})();
var e=[0,1,2,4,8,16,32,64,128,27,54],i=i.AES=h.extend({_doReset:function(){for(var c=this._key,b=c.words,d=c.sigBytes/4,c=4*((this._nRounds=d+6)+1),i=this._keySchedule=[],j=0;j<c;j++)if(j<d)i[j]=b[j];else{var h=i[j-1];j%d?6<d&&4==j%d&&(h=l[h>>>24]<<24|l[h>>>16&255]<<16|l[h>>>8&255]<<8|l[h&255]):(h=h<<8|h>>>24,h=l[h>>>24]<<24|l[h>>>16&255]<<16|l[h>>>8&255]<<8|l[h&255],h^=e[j/d|0]<<24);i[j]=i[j-d]^h}b=this._invKeySchedule=[];for(d=0;d<c;d++)j=c-d,h=d%4?i[j]:i[j-4],b[d]=4>d||4>=j?h:k[l[h>>>24]]^f[l[h>>>
16&255]]^g[l[h>>>8&255]]^a[l[h&255]]},encryptBlock:function(a,b){this._doCryptBlock(a,b,this._keySchedule,o,m,s,n,l)},decryptBlock:function(c,b){var d=c[b+1];c[b+1]=c[b+3];c[b+3]=d;this._doCryptBlock(c,b,this._invKeySchedule,k,f,g,a,r);d=c[b+1];c[b+1]=c[b+3];c[b+3]=d},_doCryptBlock:function(a,b,d,e,f,h,i,g){for(var l=this._nRounds,k=a[b]^d[0],m=a[b+1]^d[1],o=a[b+2]^d[2],n=a[b+3]^d[3],p=4,r=1;r<l;r++)var s=e[k>>>24]^f[m>>>16&255]^h[o>>>8&255]^i[n&255]^d[p++],u=e[m>>>24]^f[o>>>16&255]^h[n>>>8&255]^
i[k&255]^d[p++],v=e[o>>>24]^f[n>>>16&255]^h[k>>>8&255]^i[m&255]^d[p++],n=e[n>>>24]^f[k>>>16&255]^h[m>>>8&255]^i[o&255]^d[p++],k=s,m=u,o=v;s=(g[k>>>24]<<24|g[m>>>16&255]<<16|g[o>>>8&255]<<8|g[n&255])^d[p++];u=(g[m>>>24]<<24|g[o>>>16&255]<<16|g[n>>>8&255]<<8|g[k&255])^d[p++];v=(g[o>>>24]<<24|g[n>>>16&255]<<16|g[k>>>8&255]<<8|g[m&255])^d[p++];n=(g[n>>>24]<<24|g[k>>>16&255]<<16|g[m>>>8&255]<<8|g[o&255])^d[p++];a[b]=s;a[b+1]=u;a[b+2]=v;a[b+3]=n},keySize:8});p.AES=h._createHelper(i)})();
o(︶︿︶)o 唉,我X,终于写到这地方了。一只手打字就是累。。。
希望此贴能给朋友们带来帮助。。。排版很渣,请无视。
Python爬虫抓取某音乐网站MP3(下载歌曲、存入Sqlite)的更多相关文章
-
python 爬虫抓取心得
quanwei9958 转自 python 爬虫抓取心得分享 urllib.quote('要编码的字符串') 如果你要在url请求里面放入中文,对相应的中文进行编码的话,可以用: urllib.quo ...
-
Python爬虫----抓取豆瓣电影Top250
有了上次利用python爬虫抓取糗事百科的经验,这次自己动手写了个爬虫抓取豆瓣电影Top250的简要信息. 1.观察url 首先观察一下网址的结构 http://movie.douban.com/to ...
-
Python爬虫抓取东方财富网股票数据并实现MySQL数据库存储
Python爬虫可以说是好玩又好用了.现想利用Python爬取网页股票数据保存到本地csv数据文件中,同时想把股票数据保存到MySQL数据库中.需求有了,剩下的就是实现了. 在开始之前,保证已经安装好 ...
-
python爬虫抓取哈尔滨天气信息(静态爬虫)
python 爬虫 爬取哈尔滨天气信息 - http://www.weather.com.cn/weather/101050101.shtml 环境: windows7 python3.4(pip i ...
-
python爬虫 抓取一个网站的所有网址链接
sklearn实战-乳腺癌细胞数据挖掘 https://study.163.com/course/introduction.htm?courseId=1005269003&utm_campai ...
-
Python 爬虫: 抓取花瓣网图片
接触Python也好长时间了,一直没什么机会使用,没有机会那就自己创造机会!呐,就先从爬虫开始吧,抓点美女图片下来. 废话不多说了,讲讲我是怎么做的. 1. 分析网站 想要下载图片,只要知道图片的地址 ...
-
Python爬虫爬取美剧网站
一直有爱看美剧的习惯,一方面锻炼一下英语听力,一方面打发一下时间.之前是能在视频网站上面在线看的,可是自从广电总局的限制令之后,进口的美剧英剧等貌似就不在像以前一样同步更新了.但是,作为一个宅diao ...
-
Python爬虫抓取 python tutorial中文版,保存为word
看到了中文版的python tutorial,发现是网页版的,刚好最近在学习爬虫,想着不如抓取到本地 首先是网页的内容 查看网页源码后发现可以使用BeautifulSoup来获取文档的标题和内容,并保 ...
-
Python爬虫 -- 抓取电影天堂8分以上电影
看了几天的python语法,还是应该写个东西练练手.刚好假期里面看电影,找不到很好的影片,于是有个想法,何不搞个爬虫把电影天堂里面8分以上的电影爬出来.做完花了两三个小时,撸了这么一个程序.反正蛮简单 ...
随机推荐
-
Linux Running State Process ";.so";、";code"; Injection Technology
catalog . 引言 . 基于so文件劫持进行代码注入 . 基于函数符号表(PLT)中库函数入口地址的修改进行代码注入 . PLT redirection through shared objec ...
-
输入格式--InputFormat和InputSplit
1)InputFormat的类图: InputFormat 直接子类有三个:DBInputFormat.DelegatingInputFormat和FileInputFormat,分别表示输入文件的来 ...
-
[ASP.NET] 图形验证码破解-以简单图形为例
原文 http://www.dotblogs.com.tw/joysdw12/archive/2013/06/08/captcha-cracked.aspx 前言 这次来讲个比较有趣的主题,就是该如何 ...
-
Sizzle一步步实现所有功能(基本筛选)
第二步:实现:first,:last,:eq(),even,odd,:gt(),:lt(); :header,:root,:taget; :not(). ;(function( window ){ v ...
- HTML表单属性集合
-
ruby:TypeError: 对象不支持此属性或方法
解决办法. 1.下载对应版本 下载node.js,根据ruby版本决定下载32还是x64,我的ruby版本x64 https://npm.taobao.org/mirrors/node/v8.9.3/ ...
-
Java之Collections.emptyList()、emptySet()、emptyMap()的作用和好处以及要注意的地方
转自https://www.cnblogs.com/qiumingcheng/p/7126281.html 先说明一下好处有哪些:1,如果你想 new 一个空的 List ,而这个 List 以后也不 ...
-
VUE-007-通过路由 router.push 传递 query 参数(路由 name 识别,请求链接显示参数传递)
在前端页面表单列表修改时,经常需要在页面切换的时候,传递需要修改的表单内容,通常可通过路由进行表单参数的传递. 首先,配置页面跳转路由.在 router/index.js 中配置相应的页面跳转路由,如 ...
-
Linux - sort 排序
-t # 指定排序时所用的栏位分隔字符 -n # 依照数值的大小排序 -r # 以相反的顺序来排序 -f # 排序时,将小写字母视为大写字母 -d # 排序时,处理英文字母.数字及空格字符外,忽略其他 ...
-
[LintCode] Linked List Cycle(带环链表)
描述 给定一个链表,判断它是否有环. 样例 给出 -21->10->4->5, tail connects to node index 1,返回 true. 这里解释下,题目的意思, ...