Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

爬取微博时只获取了前几页内容就自动停止了，提示'NoneType' object has no attribute 'xpath' #544

Open
shiverwang76 opened this issue Nov 10, 2023 · 3 comments
Open

爬取微博时只获取了前几页内容就自动停止了，提示'NoneType' object has no attribute 'xpath' #544

shiverwang76 opened this issue Nov 10, 2023 · 3 comments
Labels
failed 程序运行出错 wontfix

Comments

@shiverwang76
Copy link

为了更好的解决问题，请认真回答下面的问题。等到问题解决，请及时关闭本issue。

  • 问：请您指明哪个版本运行出错（github版/PyPi版/全部）？

答：github版

  • 问：您使用的是否是最新的程序（是/否）？

答：是

  • 问：爬取任意用户都会运行出错吗（是/否）？

答：不确定

  • 问：若只有爬特定微博时才出错，能否提供出错微博的weibo_id或url（非必填）？

答：不确定

  • 问：若您已提供出错微博的weibo_id或url，可忽略此内容，否则能否提供出错账号的user_id及您配置的since_date，方便我们定位出错微博（非必填）？

答：

  • 问：如果方便，请您描述出错详情，最好附上错误提示。

答：------------------------------已获取胡锡进(1989660417)的第14页微博------------------------------
10条微博写入csv文件完毕，保存路径：E:\weiboSpider\weibo\胡锡进\1989660417.csv
10条微博写入txt文件完毕，保存路径：E:\weiboSpider\weibo\胡锡进\1989660417.txt
Progress: 1%|▊ | 14/1245 [06:31<9:34:08, 27.98s/it]
'NoneType' object has no attribute 'xpath'
Traceback (most recent call last):
File "E:\weiboSpider\weibo_spider\spider.py", line 179, in get_weibo_info
weibos, self.weibo_id_list, to_continue = PageParser(
File "E:\weiboSpider\weibo_spider\parser\page_parser.py", line 47, in init
info = self.selector.xpath("//div[@Class='c']")
AttributeError: 'NoneType' object has no attribute 'xpath'
共爬取127条原创微博
信息抓取完毕
@shiverwang76 shiverwang76 added the failed 程序运行出错 label Nov 10, 2023
@dataabc
Copy link
Owner

dataabc commented Nov 10, 2023

感谢反馈。可能是被暂时限制了，如果方便，过几天再运行看看。

@SoChiChung
Copy link

SoChiChung commented Nov 22, 2023 •
edited 

'NoneType' object has no attribute 'nickname'
Traceback (most recent call last):
  File "C:\Users\lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\weibo_spider\spider.py", line 226, in _get_filepath
    dir_name = self.user.nickname
AttributeError: 'NoneType' object has no attribute 'nickname'
'NoneType' object has no attribute '__dict__'
Traceback (most recent call last):
  File "C:\Users\lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\weibo_spider\spider.py", line 313, in get_one_user
    self.write_user(self.user)
  File "C:\Users\lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\weibo_spider\spider.py", line 137, in write_user
    writer.write_user(user)
  File "C:\Users\lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\weibo_spider\writer\txt_writer.py", line 29, in write_user
    [v + '：' + str(self.user.__dict__[k]) for k, v in self.user_desc])
  File "C:\Users\lenovo\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\weibo_spider\writer\txt_writer.py", line 29, in <listcomp>
    [v + '：' + str(self.user.__dict__[k]) for k, v in self.user_desc])
AttributeError: 'NoneType' object has no attribute '__dict__'

我从断点的日期开始爬后 报了这个错误 我也很疑惑

但是过一会我重新运行程序 又好了 可以是冷却时间不够

@stale Stale
Copy link

stale bot commented Mar 17, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Mar 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
failed 程序运行出错 wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dataabc @SoChiChung @shiverwang76