關於網易雲音樂爬蟲開發和API的一個小總結
這學期初做了一下網易雲音樂的數據採集,主要用到了python爬蟲和網易雲音樂相關的API。
今天看了@董偉明 的文章,感覺自己寫的爬蟲還有很多不足,也打算仿照文中的方法把數據還有分析結果整合一下前端,搞一個web出來。主要是看到了文章下面一些小夥伴的問題後,感覺很多問題都是自己曾經踩過的坑。突發奇想,產生衝動,在這裡總結一下,並且聊一聊自己開發網易雲音樂爬蟲以及使用它們API的一些經歷和一些經驗。
#######我的第一個網易雲音樂爬蟲
在這一個版本中,我的目的是要獲得儘可能多的中文歌詞,所以我的計劃是爬取網易雲音樂上的全部華語歌手(包括男歌手、女歌手以及組合)的全部專輯中全部歌曲的歌詞,每一篇歌詞單獨存儲為一個.txt文件。即思路是歌手->專輯->歌曲->歌詞。在這裡,通過對API以及相關網頁的分析,我發現,除了歌手id我需要自己寫爬蟲來獲得以外,其他信息的全部可以通過API來獲得。這是極好的,尤其是對於一個爬蟲beginner來說,而且網易雲音樂的頁面有充斥著js,用phantomjs效率很低,而且還沒有開始寫多線程
關於數據的存儲,除了歌詞文件本身以.txt文件存儲外,其他數據都保存在mysql資料庫里。
有必要先說一下資料庫設計(mysql),資料庫包含三個表
artist_info(保存歌手信息,包括歌手id和歌手名字)
album_info(保存專輯信息,包括專輯id和專輯名字,專輯發布時間以及外鍵artist_id關聯artist_info表的artist_id)
music_info(保存音樂信息外鍵artist_id關聯artist_info表的artist_id,外鍵album_id關聯album_info表的album_id)
#############好了,正式開始了,先說爬虫部分。
爬虫部分用來獲得全部歌手id即artist_id。
使用scrapy框架。
目錄結構,這裡使用標準scrapy文件結構,關於scrapy框架我就不過多介紹了,才疏學淺,現在只是處於會基本使用的階段而已:
1、分析網頁。打開網易雲音樂網頁版,與我的需要有關的URL結構是這樣的:
歌手列表頁
http://music.163.com/#/discover/artist/cat?id=1001&initial=65華語男歌手頁A開頭
http://music.163.com/#/discover/artist/cat?id=1002&initial=65華語女歌手頁A開頭
http://music.163.com/#/discover/artist/cat?id=1003&initial=65華語組合/樂隊頁A開頭
id取[1001,1002,1003]分別表示男歌手、女歌手和華語組合/樂隊;initial欄位可以取0或者[65-90]之間的任意數字,分別表示以特殊字元開頭或者A-Z開頭歌手列表。
首先分析了一下上述的頁面,隨便打開一個嘗試一下,採用python最基本的urllib2:
import urllib2url = "http://music.163.com/#/discover/artist/cat?id=1001&initial=65"request = urllib2.Request(url)request.add_header(User-Agent, Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36)response = urllib2.urlopen(request)result = response.read()print result
列印的結果是有js渲染的頁面,我這裡的處理方案是使用phantomJS。把它添加到middleware.py(scrapy框架下自動生成的)文件里
# -*- coding: utf-8 -*-from selenium import webdriverfrom scrapy.http import HtmlResponseimport timeclass JavaScriptMiddleware(object): def process_request(self, request, spider): if spider.name == "artists_spider": print "PhantomJS is starting..." driver = webdriver.PhantomJS() #指定使用的瀏覽器 driver.get(request.url) driver.switch_to.frame(driver.find_element_by_xpath("//iframe")) content = driver.page_source return HtmlResponse(driver.current_url, body=content,encoding=utf-8, request=request) else: return
2、編寫items.py(model,與資料庫中的欄位對應,即artist_id和artist_name)
# -*- coding: utf-8 -*-# Define here the models for your scraped items## See documentation in:# http://doc.scrapy.org/en/latest/topics/items.htmlimport scrapy#artist的itemclass MusicSpiderItem(scrapy.Item): # define the fields for your item here like: artist_id = scrapy.Field() artist_name = scrapy.Field() pass
3、編寫pipelines.py(這裡用來進行插入資料庫的操作)
# -*- coding: utf-8 -*-# Define your item pipelines here## Dont forget to add your pipeline to the ITEM_PIPELINES setting# See: http://doc.scrapy.org/en/latest/topics/item-pipeline.htmlfrom twisted.enterprise import adbapiimport mysql.connectorfrom scrapy.crawler import Settings as settingsfrom music_spider.items import MusicSpiderItemclass MusicSpiderPipeline(object): def __init__(self): try: self.db_config = { user: root, password: root, host: 127.0.0.1, database: spiderdb } self.db = mysql.connector.connect(**self.db_config) self.cursor = self.db.cursor() except mysql.connector.Error as err: print(錯誤: 連接資料庫失敗,原因%d: %s % (err.args[0], err.args[1])) #the default pipeline invoke function def process_item(self, item, spider): if isinstance(item, MusicSpiderItem): self.insert_into_table(item) else: pass return item def insert_into_table(self, item): insert_sql = ("INSERT INTO artist_info (artist_id, artist_name) VALUES (%s, %s)") data = (item[artist_id],item[artist_name]) try: result = self.cursor.execute(insert_sql, data) self.db.commit() print(保存歌手信息成功.) except mysql.connector.Error as err: self.db.rollback() print("錯誤: 插入數據失敗,原因%d, %s" % (err.args[0], err.args[1]))
4、編寫artist_spider.py
這裡使用了xpath對或得到的html頁面進行解析,提取出artist_id和artist_name並寫到資料庫中
# -*- coding: utf-8 -*-import scrapyfrom scrapy.http.request import Requestfrom music_spider.items import MusicSpiderItemfrom scrapy.selector import Selectorimport reclass ArtistSpider(scrapy.Spider): name = "artists_spider" allowed_domains = ["163.com"] def start_requests(self): for i in range(1001,1004): for j in range(65,91): #獲得A-Z歌手信息 yield Request("http://music.163.com/#/discover/artist/cat?id=%d&initial=%d" %(i,j),callback=self.parse,dont_filter=True) #獲得 「其他」 歌手信息 yield Request("http://music.163.com/#/discover/artist/cat?id=%d&initial=0" %i, callback=self.parse,dont_filter=True) #處理華語男歌手、華語女歌手以及華語樂隊組合頁面 def parse(self, response): #開始分析頁面 抽取出歌手artist_id和artist_name artist_info = MusicSpiderItem() sel = Selector(response) artist_ids = sel.xpath(//a[contains(@href, "/artist?id=")][@class="nm nm-icn f-thide s-fc0"]).re(r/artist?id=s*(d*)") artist_names = sel.xpath(//a[contains(@href, "/artist?id=")][@class="nm nm-icn f-thide s-fc0"]/text()).extract() for i in range(len(artist_ids)): artist_info[artist_id] =artist_ids[i].encode("utf-8") artist_info[artist_name] =artist_names[i].encode("utf-8") yield artist_infoif __name__ == __main__: a = ArtistSpider() a.parse()
#############再說一下API部分
先介紹一下我這裡用到的幾個API,
(歌曲id的獲取我還沒有說,那部分api我先空著,有空再寫,有興趣的同學可以打開網易雲音樂網頁版,隨便進入一首歌http://music.163.com/#/song?id=29759733,這個id就是歌曲id,可以先用這個玩一下)
1、根據歌曲id獲得歌詞:
GEThttp://music.163.com/api/song/lyric?os=pc&id=29759733&lv=-1&kv=-1&tv=-1
參數:
id:表示歌曲id
lv:表示lrc格式
kv和tv不明
結果:
{ "sgc": false, "sfy": false, "qfy": false, "lyricUser": { "id": 29759733, "status": 0, "demand": 0, "userid": 84338674, "nickname": "烏龜愛手指", "uptime": 1440407414540 }, "lrc": { "version": 6, "lyric": "[by:烏龜愛手指]
[00:31.12]可惜在遇見我那天你並不快樂
[00:38.00]可能是因為我們相遇的太晚了
[00:45.47]可是我要走了 可溫暖要走了
[00:53.00]可否有另一個我在你身後給予快樂
[02:41.54][01:01.18]可當我牽著你的手傻乎乎的樂
[02:49.10][01:07.85]渴望的愛情終於在我生命出現了
[02:56.70][01:15.36]可時間倒數了 可你的答案停住了
[03:03.97][01:22.81]可想到你的臉我還是很快樂
[04:23.09][03:11.17][01:29.90]可能你不快樂 可惜你不快樂
[04:30.02][03:18.68][01:37.66]可能是我的愛情它來的太晚了
[04:37.88][03:26.54][01:45.38]可他給了你些什麼 你是不是真快樂
[04:45.81][03:34.22][01:52.87]可要聽我的話別再為他犯傻了
[04:52.22][03:40.80][01:59.70]可能你不快樂 可我要你快樂
[04:59.88][03:48.65][02:07.16]可能是我的愛情它來的太晚了
[05:07.72][03:56.27][02:15.34]可我只想對你說 我絕對不退出了
[05:15.44][04:03.97][02:22.82]可以讓你快樂是我的快樂
[04:17.30][02:33.69]
" }, "klyric": { "version": 0 }, "tlyric": { "version": 0, "lyric": null }, "code": 200}
2、根據歌曲id獲得評論:
POSThttp://music.163.com/weapi/v1/resource/comments/R_SO_4_29759733/參數:{ "username": "", "rememberLogin": "true", "password": "", "limit": "20", "offset": "1"}
R_SO_4_後面的一串數字即歌曲id
有必要說一下,這裡的limit表示返回的評論數量(上限為20,默認也是20),offset表示評論的頁碼,例如limit=20,offset=1表示從第一個評論開始的20條評論即第1-20條評論,offset=1時,返回的結果包含熱門評論(網易雲音樂設定熱門評論最大數目為10);limit=20,offset=20表示從第21個評論開始的20條評論即第21-40條評論,依此類推,網易雲音樂的limit和offset都是這種用法,不僅僅是API中,在網頁上也是一樣。這一部分我在網上一直沒搜到,是憑藉自己對網易雲音樂url構造的經驗推測出來的,實驗結果是正確的,以供大家參考。
不同於其他api,這裡的參數是需要進行加密的,參考網上搜的加密演算法,用Java寫的(是舶來品,但是實在找不到出處了)
import org.apache.commons.codec.binary.Hex;import org.json.JSONObject;import org.jsoup.Jsoup;import org.jsoup.nodes.Document;import javax.crypto.Cipher;import javax.crypto.spec.IvParameterSpec;import javax.crypto.spec.SecretKeySpec;import java.io.IOException;import java.math.BigInteger;import java.util.ArrayList;import java.util.Base64;public class Main { //AES加密 public static String encrypt(String text, String secKey) throws Exception { byte[] raw = secKey.getBytes(); SecretKeySpec skeySpec = new SecretKeySpec(raw, "AES"); // "演算法/模式/補碼方式" Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding"); // 使用CBC模式,需要一個向量iv,可增加加密演算法的強度 IvParameterSpec iv = new IvParameterSpec("0102030405060708".getBytes()); cipher.init(Cipher.ENCRYPT_MODE, skeySpec, iv); byte[] encrypted = cipher.doFinal(text.getBytes()); return Base64.getEncoder().encodeToString(encrypted); } //字元填充 public static String zfill(String result, int n) { if (result.length() >= n) { result = result.substring(result.length() - n, result.length()); } else { StringBuilder stringBuilder = new StringBuilder(); for (int i = n; i > result.length(); i--) { stringBuilder.append("0"); } stringBuilder.append(result); result = stringBuilder.toString(); } return result; } public void commentAPI(String text,String music_id){ //私鑰,隨機16位字元串(自己可改) String secKey = "cd859f54539b24b7"; String modulus = "00e0b509f6259df8642dbc35662901477df22677ec152b5ff68ace615bb7b725152b3ab17a876aea8a5aa76d2e417629ec4ee341f56135fccf695280104e0312ecbda92557c93870114af6c9d05c4f7f0c3685b7a46bee255932575cce10b424d813cfe4875d3e82047b97ddef52741d546b8e289dc6935b3ece0462db0a22b8e7"; String nonce = "0CoJUm6Qyw8W8jud"; String pubKey = "010001"; try { //2次AES加密,得到params String params = EncryptTools.encrypt(EncryptTools.encrypt(text, nonce), secKey); StringBuffer stringBuffer = new StringBuffer(secKey); //逆置私鑰 secKey = stringBuffer.reverse().toString(); String hex = Hex.encodeHexString(secKey.getBytes()); BigInteger bigInteger1 = new BigInteger(hex, 16); BigInteger bigInteger2 = new BigInteger(pubKey, 16); BigInteger bigInteger3 = new BigInteger(modulus, 16); //RSA加密計算 BigInteger bigInteger4 = bigInteger1.pow(bigInteger2.intValue()).remainder(bigInteger3); String encSecKey= Hex.encodeHexString(bigInteger4.toByteArray()); //字元填充 encSecKey= EncryptTools.zfill(encSecKey, 256); //評論獲取 Document document = Jsoup.connect("http://music.163.com/weapi/v1/resource/comments/R_SO_4_"+music_id+"/").cookie("appver", "1.5.0.75771") .header("Referer", "http://music.163.com/").data("params", params).data("encSecKey", encSecKey) .ignoreContentType(true).post(); JSONObject json=new JSONObject(document.text()); System.out.println(json); } catch (IOException e1) { e1.printStackTrace(); } catch (Exception e) { e.printStackTrace(); } } public static void main(String[] args){ Main m = new Main(); ArrayList<String> as = new ArrayList<>(); String music_id = "437859519"; String text = "{"username": "", "rememberLogin": "true", "password": "" , "limit": "20", "offset": "20"}"; m.commentAPI(text,music_id); }}
json結果:
下面這是一個包含hotComments(熱門評論)的結果,Obviously,這裡的offset我設置為1啦。所以熱門評論(hotComments)+普通評論(comments)=20
如果offset=20那麼就沒有hotComments這個key啦,因為網易雲音樂的熱門評論最多有10條哦! 在這裡還可以獲得到總評論數key="total",關於json結構我就不進行進一步解析了,相信大家都很熟。
{ "isMusician": false, "userId": -1, "topComments": [ ], "moreHot": true, "hotComments": [ { "user": { "locationInfo": null, "userId": 31188374, "expertTags": null, "authStatus": 0, "nickname": "若是歸來去", "avatarUrl": "http://p4.music.126.net/EoddBg5e-PKal56BV1mq9g==/3311729029883325.jpg", "remarkName": null, "vipType": 10, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 23275, "commentId": 238472796, "time": 1477836375609, "content": "華晨宇,創作界的鬼才!" }, { "user": { "locationInfo": null, "userId": 82179192, "expertTags": [ "流行", "華語", "韓語" ], "authStatus": 0, "nickname": "艾琳艾德勒丶", "avatarUrl": "http://p4.music.126.net/OaxBBPSYo-_WZ1RQ-Wn_tw==/1415071470460270.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 18252, "commentId": 238472923, "time": 1477837304066, "content": "把滑板鞋改的這麼洋氣也是沒誰了。" }, { "user": { "locationInfo": null, "userId": 324934319, "expertTags": null, "authStatus": 0, "nickname": "逆風而行的旅人", "avatarUrl": "http://p4.music.126.net/EB7i8Db2LDEb52p2lcxGOA==/3412884131796231.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 12694, "commentId": 238499804, "time": 1477836981760, "content": "真是路轉粉啊 然後打算讓我媽也粉上花花的我給我媽聽了原唱 然後我媽也路轉粉了還一邊念叨: 這小夥子原來不止臉蛋子帥啊 哈哈哈[大哭]" }, { "user": { "locationInfo": null, "userId": 44292263, "expertTags": null, "authStatus": 0, "nickname": "和2億有趣的村民一起聽歌看評論", "avatarUrl": "http://p4.music.126.net/2qOffAQK3soED5l7eL09BA==/3250156386606135.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 9680, "commentId": 238551606, "time": 1477837148119, "content": "花花城會玩,這改編厲害了,瞬間感覺很高大上,完全找不到「摩擦,摩擦」的調調了,在舞台上你就是國王,燃炸天,瞬間被你的才華折服。[愛心]" }, { "user": { "locationInfo": null, "userId": 88278295, "expertTags": null, "authStatus": 0, "nickname": "念珠散", "avatarUrl": "http://p3.music.126.net/a-crySgFhZ_6Rdigeahr8Q==/1384285156404178.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 8363, "commentId": 238542863, "time": 1477837778263, "content": "他是新生代歌手中很有實力的,毋庸置疑,24小時內不是改編而且作曲,看了這場live的視頻,更能感受到他的魅力。很棒啊,好喜歡" }, { "user": { "locationInfo": null, "userId": 350545021, "expertTags": null, "authStatus": 0, "nickname": "荷包蛋yu", "avatarUrl": "http://p4.music.126.net/wnuRADemgtGF6EHxhECOZw==/18745573742730703.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 8097, "commentId": 238531939, "time": 1477838448843, "content": "[cp]一般唱情歌的不會搖滾,唱搖滾的不說Rap,說Rap的不唱民謠,唱民謠的不玩實驗,玩實驗的不寫古典。。可是華晨宇都會啊,好可怕。。。[生病][生病][生病]" }, { "user": { "locationInfo": null, "userId": 76158939, "expertTags": null, "authStatus": 0, "nickname": "一蕊三心", "avatarUrl": "http://p3.music.126.net/EMjGq5TjBHqYz7JIrAsSCw==/1365593505137136.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 7716, "commentId": 238597603, "time": 1477843078962, "content": "他唱什麼都貴貴的感覺。。。故鄉的雲感覺是賺得盆滿缽滿衣錦還鄉,滑板鞋。。。這尼瑪應該是限量吧。。" }, { "user": { "locationInfo": null, "userId": 125832624, "expertTags": null, "authStatus": 0, "nickname": "你所願", "avatarUrl": "http://p3.music.126.net/tE2cnH6LiSoTG7S03nfslw==/1371090999917385.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 7058, "commentId": 238559658, "time": 1477841831322, "content": "24小時內都做了什麼:華晨宇聽原曲確定改編思路→華晨宇練習rap→華晨宇作曲→鄭楠編曲→華晨宇拿到編曲後再練習→華晨宇背歌詞rap→綵排→合樂隊→根據歌曲做造型→登台 [攤手] 24小時並不是純作曲而已,希望大家明白,你的偶像,遠比你知道的還要厲害。" }, { "user": { "locationInfo": null, "userId": 124861944, "expertTags": null, "authStatus": 0, "nickname": "吃椰子不吐椰子皮兒_", "avatarUrl": "http://p4.music.126.net/RFACoULOHViSDUON8X6GWQ==/1405175869314771.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 6568, "commentId": 238612077, "time": 1477839472489, "content": "路人聽了這首歌表示。。。內地樂壇可能要被一個叫華晨宇的人震動了" }, { "user": { "locationInfo": null, "userId": 262049149, "expertTags": null, "authStatus": 0, "nickname": "摸摸paul", "avatarUrl": "http://p4.music.126.net/jm7Uy4UNqcJ2PlG2vaJRhA==/1420569028503444.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 5067, "commentId": 238478995, "time": 1477837823727, "content": "越是神曲,越是有突破性的創造。借用楊坤的話,後生可畏!華晨宇從《流浪記》的驚艷,到《我的滑板鞋》的綻放,才華終將閃光![親親]" } ], "code": 200, "comments": [ { "user": { "locationInfo": null, "userId": 63037343, "expertTags": null, "authStatus": 0, "nickname": "AndyWangYH", "avatarUrl": "http://p4.music.126.net/hggem_lj1rO9i3Wiv3zHfg==/2884019002420845.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246471574, "time": 1479082745510, "content": "看來不是歌詞不行太俗", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 361633617, "expertTags": null, "authStatus": 0, "nickname": "咭羋Dooley霞", "avatarUrl": "http://p3.music.126.net/ziQ7wWBhRZ-02w7R5ZVF0Q==/3402988506994332.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246463619, "time": 1479082676000, "content": "一個字 帥[親親][親親][親親]", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 311737776, "expertTags": null, "authStatus": 0, "nickname": "毛病深成", "avatarUrl": "http://p3.music.126.net/tm0puVXmiMzNV7kiejXqAg==/18727981556623295.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246477482, "time": 1479082507573, "content": "好多水軍", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 100614922, "expertTags": null, "authStatus": 0, "nickname": "Min-Crazy夏sir", "avatarUrl": "http://p3.music.126.net/tOfSCC9_RZT2K5GsLFnn_Q==/3428277256962417.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246432653, "time": 1479082495515, "content": "不是水軍@x@。當初龐麥郎出滑板鞋這首歌的時候各位聽客看客又都是什麼態度~呵呵噠~不知道背後噴人家多少次~笑話人家多少次了吧~現在華晨宇同學改編~一大部分吃瓜群眾又跟著噴改編了~什麼心態⊙▽⊙。不過人各所愛~觀眾老爺眾口難調…但是留點兒口德還是好的。", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 360199490, "expertTags": null, "authStatus": 0, "nickname": "即相識不相忘", "avatarUrl": "http://p3.music.126.net/IRsPkjQACczf1Od7FyZu3A==/18550960184934718.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246466543, "time": 1479082237497, "content": "真是聽得雞皮疙瘩起來了,好聽?", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 348313592, "expertTags": null, "authStatus": 0, "nickname": "搖搖要要妖妖", "avatarUrl": "http://p3.music.126.net/H6-KrT0NQRf3cT9-WLWUpw==/1368891993286143.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 2, "commentId": 246409919, "time": 1479082183891, "content": "3w評論是水軍,100多w評論呢?海軍?真搞笑啊!", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 95894367, "expertTags": null, "authStatus": 0, "nickname": "那紛飛的年華", "avatarUrl": "http://p3.music.126.net/kV1twzrXGGw1ym4MhrLIpQ==/3274345629201687.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 2, "commentId": 246459582, "time": 1479081888779, "content": "水軍真多 二營長把老子的原子彈拉過來 我們跟水軍同歸於盡,友軍快躲躲", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 360237296, "expertTags": null, "authStatus": 0, "nickname": "可喲", "avatarUrl": "http://p4.music.126.net/s3egE3ts0MX89bNQXqPZkA==/109951162808108611.jpg", "remarkName": null, "vipType": 10, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246470551, "time": 1479081791033, "content": "很厲害!!!真的有點路轉粉了,不過原唱也很個性啊~~沒必要非貶低人家啊。。", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 96651014, "expertTags": null, "authStatus": 0, "nickname": "upbaby鹿", "avatarUrl": "http://p4.music.126.net/iAMGHoURb_smEC6o82r2gg==/2946691188226629.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246451554, "time": 1479081701551, "content": "花花好厲害", "isRemoveHotComment": false }, { "user": { "locationInfo": null, "userId": 96651014, "expertTags": null, "authStatus": 0, "nickname": "upbaby鹿", "avatarUrl": "http://p4.music.126.net/iAMGHoURb_smEC6o82r2gg==/2946691188226629.jpg", "remarkName": null, "vipType": 0, "userType": 0 }, "beReplied": [ ], "liked": false, "likedCount": 0, "commentId": 246468552, "time": 1479081696076, "content": "真的迷上這首歌了", "isRemoveHotComment": false } ], "total": 36838, "more": true}
今天先介紹這兩個API,剩下的,包括根據歌手id獲得專輯id以及根據專輯id獲得歌曲id的API我有空再碼。
p.s.下面這個鏈接是一個集成了很多網易雲音樂api的開源項目,供大家參考。
https://github.com/littlecodersh/NetEaseMusicApi
近期打算把之前在網易雲音樂上採集的數據集成一個前端,做一個小的web項目,能夠展示一些歌詞分析的結果。
推薦閱讀:
※Python安裝Scrapy出現以下錯誤怎麼辦?
※關於在pycharm中配置導入路徑的問題?
※基於Scrapy如何寫一個爬蟲抓取藝龍、攜程等網站的機票價格進行分析並做機票價格預測?
※你懂Scrapy嗎?Scrapy大型爬蟲框架講解【一】