爬取58同城，解決反爬字體加密解析問題

【導語】我們在爬取數據中，會遇到字體亂碼的下，其實是字體加密，本篇文章主要解決字體解密這種反爬方式。

1.在瀏覽器打開58同城網址進入北京租房
2.點擊檢查,找到房租價格對應位置,發現源碼中價格部分是亂碼,但是頁面顯示正常
3.得出結果這段代碼是經過字體加密的

我們看到的如下：

那麼，按F12查看源碼，如下：

【解決思路】

字體加密一般是通過字體文件進行映射,所以我們只需要找到對應的字體文件,然後通過一些手段找到他們字體對應的映射關係就可以破解字體反爬

1、當我們發現字體反爬之後,首先可以找一下是通過哪一個類進行的字體映射,因為字體映射都會在具體的css樣式上進行體現
2、我們可以先刪除掉自身的一些樣式,觀察頁面變化,當我們刪除自身樣式或者父類樣式之後頁面恢復了亂碼,就可以確定是哪一個類進行的字體加密

3、通過加密樣式尋找字體映射文件方式

字體加密一般是分為兩種
①通過base64加密的字體文件,這種文件可以通過python的base64模塊進行解密
②通過url的方式引入字體文件

以上兩種我們都可以通過搜索對應的加密樣式找到具體文件
首先可以在當前頁面中搜索是否存在對這個類的定義
如果沒有找到可以在network中全局搜索存在這個樣式的css文件
找到加密文件之後無論是哪一種加密方式都可以通過python請求獲取加密字體文件

4、獲取加密文件的代碼

通過正則表達式獲取加密文件內容
獲取到加密文件之後如果想要看裡面的內容可以通過base64進行解密,並寫入woff文件中
之後可以通過fontTools包將字體文件轉為xml進行查看

<code>
import
 re
from fontTools.ttLib 
import
 TTFont
import
 base64

import
 requests

res = requests.
get
('https: 

bs64 = re.findall(';base64,(.*?)\'\) format',res.text,re.
S
)[
0
]
print
(bs64)
/<code>

<code>
import
 re
from
 fontTools.ttLib 
import
 TTFont
import
 base64

import
 requests

res = requests.get(
'https://bj.58.com/chuzu/?utm_source=market&spm=u-2d2yxv86y3v43nkddh1.BDPCPZ_BT&PGTID=0d100000-0000-13e1-612a-4575bfeb5389&ClickID=2'
)

bs64 = re.findall(
';base64,(.*?)\'\) format'
,res.text,re.S)[
0
]
def
 
base64tofile
(bs64)
:

    res = base64.b64decode(bs64)
    f = open(
'1.woff'
, 
'wb'
)
    f.write(res)
    f.close()
    font = TTFont(
'1.woff'
)
    font.saveXML(
'1.xml'
)
base64tofile(bs64)
/<code>

打開xml進行查看時最重要的內容時cmap 這裡是映射關係
code是源碼中的內容
code 是16進制數,轉成int類型之後代表Unicode上的一個對應的字符
name 是對應的頁面中顯示的內容
name是通過座標進行繪製的圖形,座標信息在TTGlyph中展示
但是如果單單是數字加密,都可以找出期中的規律,如果是文字也進行了加密,那麼只能通過一些工具查看繪製出的字體樣式,手動的進行對應

5、 解密代碼的編輯

通過找到的對應關係進行解密代碼的編輯
通過minidom讀取xml輸入提前cmap中的對應關係 通過re模塊獲取所有的價格數據,通過編寫好的解密代碼進行解密

<code>
from
 xml.dom 
import
 minidom
name_to_num = { 
'glyph00008'
:
'7'
,
                
'glyph00006'
:
'5'
,
                
'glyph00005'
:
'4'
,
                
'glyph00004'
:
'3'
,
                
'glyph00010'
:
'9'
,
                
'glyph00003'
:
'2'
,
                
'glyph00009'
:
'8'
,
                
'glyph00001'
:
'0'
,
                
'glyph00002'
:
'1'
,
                
'glyph00007'
:
'6'
}

def
 
get_num
(string)
:

    dom = minidom.parse(
'1.xml'
)
    root = dom.documentElement
    map_ = root.getElementsByTagName( 
'cmap'
)[
0
].getElementsByTagName(
'map'
)
    code_to_name = {tag.getAttribute(
"code"
)[
2
:]:tag.getAttribute(
"name"
) 
for
 tag 
in
 map_[:]}
    print(code_to_name)
    nums = []
    
for
 i 
in
 string.split(
';'
):
        
if
 
not
 i:
            
break
        name_num = code_to_name[i[
3
:]]
        num = name_to_num[name_num]
        nums.append(num)
    
return
 
''
.join(nums)

 
get_num(
'龤驋龥龒'
)
/<code>

【測試】

<code>a = 
'AAEAAAALAIAAAwAwR1NVQiCLJXoAAAE4AAAAVE9TLzL4XQjtAAABjAAAAFZjbWFwq8V/YgAAAhAAAAIuZ2x5ZuWIN0cAAARYAAADdGhlYWQbKimFAAAA4AAAADZoaGVhCtADIwAAALwAAAAkaG10eC7qAAAAAAHkAAAALGxvY2ED7gSyAAAEQAAAABhtYXhwARgANgAAARgAAAAgbmFtZTd6VP8AAAfMAAACanBvc3QEQwahAAAKOAAAAEUAAQAABmb+ZgAABLEAAAAABGgAAQAAAAAAAAAAAAAAAAAAAAsAAQAAAAEAAN6JRZhfDzz1AAsIAAAAAADb0W86AAAAANvRbzoAAP/mBGgGLgAAAAgAAgAAAAAAAAABAAAACwAqAAMAAAAAAAIAAAAKAAoAAAD/AAAAAAAAAAEAAAAKADAAPgACREZMVAAObGF0bgAaAAQAAAAAAAAAAQAAAAQAAAAAAAAAAQAAAAFsaWdhAAgAAAABAAAAAQAEAAQAAAABAAgAAQAGAAAAAQAAAAEERAGQAAUAAAUTBZkAAAEeBRMFmQAAA9cAZAIQAAACAAUDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFBmRWQAQJR2n6UGZv5mALgGZgGaAAAAAQAAAAAAAAAAAAAEsQAABLEAAASxAAAEsQAABLEAAASxAAAEsQAABLEAAASxAAAEsQAAAAAABQAAAAMAAAAsAAAABAAAAaYAAQAAAAAAoAADAAEAAAAsAAMACgAAAaYABAB0AAAAFAAQAAMABJR2lY+ZPJpLnjqeo59kn5Kfpf//AACUdpWPmTyaS546nqOfZJ+Sn6T//wAAAAAAAAAAAAAAAAAAAAAAAAABABQAFAAUABQAFAAUABQAFAAUAAAABgAIAAcABAAKAAMACQACAAEABQAAAQYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAiAAAAAAAAAAKAACUdgAAlHYAAAAGAACVjwAAlY8AAAAIAACZPAAAmTwAAAAHAACaSwAAmksAAAAEAACeOgAAnjoAAAAKAACeowAAnqMAAAADAACfZAAAn2QAAAAJAACfkgAAn5IAAAACAACfpAAAn6QAAAABAACfpQAAn6UAAAAFAAAAAAAAACgAPgBmAJoAvgDoASQBOAF+AboAAgAA/+YEWQYnAAoAEgAAExAAISAREAAjIgATECEgERAhIFsBEAECAez+6/rs/v3IATkBNP7S/sEC6AGaAaX85v54/mEBigGB/ZcCcwKJAAABAAAAAAQ1Bi4ACQAAKQE1IREFNSURIQQ1/IgBW/6cAicBWqkEmGe0oPp7AAEAAAAABCYGJwAXAAApATUBPgE1NCYjIgc1NjMyFhUUAgcBFSEEGPxSAcK6fpSMz7y389Hym9j+nwLGqgHButl0hI2wx43iv5D+69b+pwQAAQAA/+YEGQYnACEAABMWMzI2NRAhIzUzIBE0ISIHNTYzMhYVEAUVHgEVFAAjIiePn8igu/5bgXsBdf7jo5CYy8bw/sqow/7T+tyHAQN7nYQBJqIBFP9uuVjPpf7QVwQSyZbR/wBSAAACAAAAAARoBg0ACgASAAABIxEjESE1ATMRMyERNDcjBgcBBGjGvv0uAq3jxv58BAQOLf4zAZL+bgGSfwP8/CACiUVaJlH9TwABAAD/5gQhBg0AGAAANxYzMjYQJiMiBxEhFSERNjMyBBUUACEiJ7GcqaDEx71bmgL6/bxXLPUBEv7a/v3Zbu5mswEppA4DE63+SgX42uH+6kAAAAACAAD/5gRbBicAFgAiAAABJiMiAgMzNjMyEhUUACMiABEQACEyFwEUFjMyNjU0JiMiBgP6eYTJ9AIFbvHJ8P7r1+z+8wFhASClXv1Qo4eAoJeLhKQFRj7+ov7R1f762eP+3AFxAVMBmgHjLfwBmdq8lKCytAAAAAABAAAAAARNBg0ABgAACQEjASE1IQRN/aLLAkD8+gPvBcn6NwVgrQAAAwAA/+YESgYnABUAHwApAAABJDU0JDMyFhUQBRUEERQEIyIkNRAlATQmIyIGFRQXNgEEFRQWMzI2NTQBtv7rAQTKufD+3wFT/un6zf7+AUwBnIJvaJLz+P78/uGoh4OkAy+B9avXyqD+/osEev7aweXitAEohwF7aHh9YcJlZ/7qdNhwkI9r4QAAAAACAAD/5gRGBicAFwAjAAA3FjMyEhEGJwYjIgA1NAAzMgAREAAhIicTFBYzMjY1NCYjIga5gJTQ5QICZvHD/wABGN/nAQT+sP7Xo3FxoI16pqWHfaTSSgFIAS4CAsIBDNbkASX+lf6l/lP+MjUEHJy3p3en274AAAAAABAAxgABAAAAAAABAA8AAAABAAAAAAACAAcADwABAAAAAAADAA8AFgABAAAAAAAEAA8AJQABAAAAAAAFAAsANAABAAAAAAAGAA8APwABAAAAAAAKACsATgABAAAAAAALABMAeQADAAEECQABAB4AjAADAAEECQACAA4AqgADAAEECQADAB4AuAADAAEECQAEAB4A1gADAAEECQAFABYA9AADAAEECQAGAB4BCgADAAEECQAKAFYBKAADAAEECQALACYBfmZhbmdjaGFuLXNlY3JldFJlZ3VsYXJmYW5nY2hhbi1zZWNyZXRmYW5nY2hhbi1zZWNyZXRWZXJzaW9uIDEuMGZhbmdjaGFuLXNlY3JldEdlbmVyYXRlZCBieSBzdmcydHRmIGZyb20gRm9udGVsbG8gcHJvamVjdC5odHRwOi8vZm9udGVsbG8uY29tAGYAYQBuAGcAYwBoAGEAbgAtAHMAZQBjAHIAZQB0AFIAZQBnAHUAbABhAHIAZgBhAG4AZwBjAGgAYQBuAC0AcwBlAGMAcgBlAHQAZgBhAG4AZwBjAGgAYQBuAC0AcwBlAGMAcgBlAHQAVgBlAHIAcwBpAG8AbgAgADEALgAwAGYAYQBuAGcAYwBoAGEAbgAtAHMAZQBjAHIAZQB0AEcAZQBuAGUAcgBhAHQAZQBkACAAYgB5ACAAcwB2AGcAMgB0AHQAZgAgAGYAcgBvAG0AIABGAG8AbgB0AGUAbABsAG8AIABwAHIAbwBqAGUAYwB0AC4AaAB0AHQAcAA6AC8ALwBmAG8AbgB0AGUAbABsAG8ALgBjAG8AbQAAAAIAAAAAAAD/EwB3AAAAAAAAAAAAAAAAAAAAAAAAAAAACwECAQMBBAEFAQYBBwEIAQkBCgELAQwAAAAAAAAAAAAAAAAAAAAA'
 
s
 = 
'龒鵂鵂龤'
base64tofile(a)

def str_to_16(
s
):
    code_64 = []
    
for
 i in 
s
:
         
        decode_num = 
ord
(i)
         
        num =  
hex
(decode_num)
        code_64.append(
'$%s'
%num)
    
return
 
';'
.join(code_64)

code_64 = str_to_16(
'龒鵂鵂龤'
)

get_num(code_64)
/<code>

希望本文的內容對大家的學習或者工作能帶來一定的幫助，每天進步一點點，加油