Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sourcery Starbot ⭐ refactored wkunzhi/Python3-Spider #28

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

sourcery-ai-bot
Copy link

Thanks for starring sourcery-ai/sourcery ✨ 🌟 ✨

Here's your pull request refactoring your most popular Python repo.

If you want Sourcery to refactor all your Python repos and incoming pull requests install our bot.

Review changes via command line

To manually merge these changes, make sure you're on the master branch, then run:

git fetch https://github.com/sourcery-ai-bot/Python3-Spider master
git merge --ff-only FETCH_HEAD
git reset HEAD^

Copy link
Author

@sourcery-ai-bot sourcery-ai-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Due to GitHub API limits, only the first 60 comments can be shown.

Comment on lines -111 to +115
if abs(c_pixel[0] - ic_pixel[0]) < threshold and \
abs(c_pixel[1] - ic_pixel[1]) < threshold and \
abs(c_pixel[2] - ic_pixel[2]) < threshold:
return True
return False
return (
abs(c_pixel[0] - ic_pixel[0]) < threshold
and abs(c_pixel[1] - ic_pixel[1]) < threshold
and abs(c_pixel[2] - ic_pixel[2]) < threshold
)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function LoginBli.is_pixel_similar refactored with the following changes:

if current < mid:
# 加速度为正2
a = 20
else:
# 加速度为负3
a = -30
a = 20 if current < mid else -30
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function LoginBli.get_track refactored with the following changes:

This removes the following comments ( why? ):

# 加速度为正2
# 加速度为负3

Comment on lines -51 to +53
print('标题:' + title, 'ID', cid)
print(f'标题:{title}', 'ID', cid)
page = str(item['page'])
start_url = start_url + "/?p=" + page
start_url = f"{start_url}/?p={page}"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function DownVideo.get_cid_list refactored with the following changes:

Comment on lines -68 to +76
params = 'appkey=%s&cid=%s&otype=json&qn=%s&quality=%s&type=' % (appkey, cid, quality, quality)
params = f'appkey={appkey}&cid={cid}&otype=json&qn={quality}&quality={quality}&type='
chksum = hashlib.md5(bytes(params + sec, 'utf8')).hexdigest()
url_api = 'https://interface.bilibili.com/v2/playurl?%s&sign=%s' % (params, chksum)
url_api = f'https://interface.bilibili.com/v2/playurl?{params}&sign={chksum}'
headers = {
'Referer': start_url,
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'
}
html = requests.get(url_api, headers=headers).json()
video_list = [html['durl'][0]['url']]
return video_list
return [html['durl'][0]['url']]
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function DownVideo.get_play_list refactored with the following changes:

Comment on lines -81 to +80
speed_str = " Speed: %s" % self.format_size(speed)
speed_str = f" Speed: {self.format_size(speed)}"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function DownVideo.schedule_cmd refactored with the following changes:

Comment on lines -39 to +38
result = {}
for item in offset_item:
css_class = item[0][1:]
x_offset = item[1]
y_offset = item[2]
result[css_class] = [x_offset, y_offset]
return result
return {item[0][1:]: [item[1], item[2]] for item in offset_item}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function DianPing.get_css_offset refactored with the following changes:

Comment on lines -53 to +46
svg_list = [int(size), 'https:' + url]
svg_list = [int(size), f'https:{url}']
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function DianPing.get_svg_url_dict refactored with the following changes:

Comment on lines -103 to +96
print("餐馆: {}, 点评数: {}".format(shop_name, num))
print(f"餐馆: {shop_name}, 点评数: {num}")
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function DianPing.get_comment_num refactored with the following changes:

Comment on lines -53 to +54
if 'uni' + clean_code in json_data:
return json_data['uni' + clean_code]
if f'uni{clean_code}' in json_data:
return json_data[f'uni{clean_code}']
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ParseFontClass.parse_ttf refactored with the following changes:

Comment on lines -89 to +95
with open(name + '.woff', 'wb+') as f:
f.write(requests.get('http://' + ttf_list[index]).content) # 下载写入
font = TTFont(name + '.woff')
with open(f'{name}.woff', 'wb+') as f:
f.write(requests.get(f'http://{ttf_list[index]}').content)
font = TTFont(f'{name}.woff')
uni_list = font['cmap'].tables[0].ttFont.getGlyphOrder() # 取出字形保存到uniList中
json_data = json.dumps(dict(zip(uni_list, self.FONT_LIST)), ensure_ascii=False)
self.add_hash(name, json_data)
os.remove(name + '.woff') # 用完了删掉,节省资源占用
os.remove(f'{name}.woff')
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ParseFontClass.install_ttf refactored with the following changes:

This removes the following comments ( why? ):

# 下载写入
# 用完了删掉,节省资源占用

@@ -6,6 +6,7 @@
从网页下载一个字体文件获取对应推导式,动态获取请自行拓展
"""

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 18-18 refactored with the following changes:

This removes the following comments ( why? ):

# 对应关系数量转换

self.url = 'https://v.douyin.com/' + path + '/'
self.url = f'https://v.douyin.com/{path}/'
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ParseVideo.__init__ refactored with the following changes:

Comment on lines -28 to +31
result = re.search(r'itemId: "(.*?)",[\s\S]*?uid: "(.*?)",[\s\S]*?authorName: "(.*?)",[\s\S]*?dytk: "(.*?)"',
response.text)
return result
return re.search(
r'itemId: "(.*?)",[\s\S]*?uid: "(.*?)",[\s\S]*?authorName: "(.*?)",[\s\S]*?dytk: "(.*?)"',
response.text,
)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ParseVideo.go_location refactored with the following changes:

Comment on lines -33 to +34
url = 'https://www.iesdouyin.com/web/api/v2/aweme/iteminfo/?item_ids=' + ret.group(1) + '&dytk=' + ret.group(4)
url = f'https://www.iesdouyin.com/web/api/v2/aweme/iteminfo/?item_ids={ret.group(1)}&dytk={ret.group(4)}'
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ParseVideo.go_message refactored with the following changes:

Comment on lines -26 to +30
_decrypt_result = pyDes.triple_des(key, pyDes.CBC, iv, None, pyDes.PAD_PKCS5).decrypt(data).decode('utf-8')
return _decrypt_result
return (
pyDes.triple_des(key, pyDes.CBC, iv, None, pyDes.PAD_PKCS5)
.decrypt(data)
.decode('utf-8')
)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function TripleDesUtils.decrypt refactored with the following changes:

Comment on lines -133 to +131
if None == pubkey:
if pubkey is None:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function UseRSA.check_sign refactored with the following changes:

Comment on lines -171 to +169
en = k.encrypt(text, padmode=PAD_PKCS5)
return en
return k.encrypt(text, padmode=PAD_PKCS5)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function UseDES.encrypt refactored with the following changes:

Comment on lines -220 to +217
cipher_text = de.encrypt(text)
return cipher_text
return de.encrypt(text)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function UseDES3.encrypt refactored with the following changes:

Comment on lines -226 to +222
st = str(plain_text.decode("utf-8")).rstrip('\0')
return st
return str(plain_text.decode("utf-8")).rstrip('\0')
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function UseDES3.decrypt refactored with the following changes:

Comment on lines -251 to +246
encrypts = sha.hexdigest()
return encrypts
return sha.hexdigest()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function use_sha refactored with the following changes:

sign = js.call('get_pwd', pwd, publickey_mod, publickey_exp)
return sign
return js.call('get_pwd', pwd, publickey_mod, publickey_exp)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function main refactored with the following changes:

msg = json.loads(response.content)
return msg
return json.loads(response.content)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ZGC.login refactored with the following changes:

Comment on lines -73 to +76
text = _str + 'zol'
text = f'{_str}zol'
# 创建md5对象
m = hashlib.md5()
m.update(text.encode(encoding='utf-8'))
str_md5 = m.hexdigest()
return str_md5
return m.hexdigest()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function ZGC.make_md5 refactored with the following changes:

url = 'https://open.e.189.cn/api/logbox/oauth2/unifyAccountLogin.do?sign=' + ret.group(1) + '&appId=' + ret.group(
2) + '&paras=' + ret.group(3) + '&format=' + ret.group(4) + '&clientType=' + ret.group(
5) + '&version=' + ret.group(6)
url = f'https://open.e.189.cn/api/logbox/oauth2/unifyAccountLogin.do?sign={ret.group(1)}&appId={ret.group(2)}&paras={ret.group(3)}&format={ret.group(4)}&clientType={ret.group(5)}&version={ret.group(6)}'
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function login refactored with the following changes:

_key = self.js.call('get_key_iv', ret)
return _key
return self.js.call('get_key_iv', ret)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function MakeParam.get_key_vi refactored with the following changes:

Comment on lines -37 to -42
data = {
return {
'account': self.user,
'password': self.make_pwd(),
'returnUrl': '/'
'returnUrl': '/',
}
return data
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function YX.make_data refactored with the following changes:

@@ -8,13 +8,14 @@
需要 V8 引擎!
"""

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines 17-17 refactored with the following changes:

tk = "tk0.29" + str(random.randint(104771190122337, 904771190122337)) + str(_time * 1000)
tk = f"tk0.29{random.randint(104771190122337, 904771190122337)}{str(_time * 1000)}"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function BaiDu.make_dv refactored with the following changes:

Comment on lines -72 to +76
response = requests.get('https://passport.baidu.com/cgi-bin/genimage?' + verify_str, cookies=self.cookies,
headers=headers_img)
response = requests.get(
f'https://passport.baidu.com/cgi-bin/genimage?{verify_str}',
cookies=self.cookies,
headers=headers_img,
)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function BaiDu.get_img refactored with the following changes:

Comment on lines -118 to +121
url = 'https://passport.baidu.com/v2/sapi/authwidgetverify?authtoken=' + parse.quote(
token) + '&type=&jsonp=1&apiver=v3&verifychannel=&action=getapi&vcode=&questionAndAnswer=&needsid=&rsakey=&countrycode=&u=https%3A%2F%2Fpassport.qatest.baidu.com%2F%3Fgetpassresetpwd&tpl=&winsdk=&authAction=&callback=bd__cbs__oeq73u'
url = f'https://passport.baidu.com/v2/sapi/authwidgetverify?authtoken={parse.quote(token)}&type=&jsonp=1&apiver=v3&verifychannel=&action=getapi&vcode=&questionAndAnswer=&needsid=&rsakey=&countrycode=&u=https%3A%2F%2Fpassport.qatest.baidu.com%2F%3Fgetpassresetpwd&tpl=&winsdk=&authAction=&callback=bd__cbs__oeq73u'
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function BaiDu.get_phone refactored with the following changes:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant