- 1.代码基于python的scrapy爬虫框架。爬取url保存在Redis,爬取数据主要保存在MongoDB
- 2.依赖模块有pymongo，scrapy-redis，scrapy，redis，通过以下命令安装：
$ pip install pymongo scrapy scrapy-redis redis
- 使用方法:下载项目以后，进入项目根文件夹，运行：
$ scrapy crawl toutiao
forked from Harhao/toutiao
-
Notifications
You must be signed in to change notification settings - Fork 0
odydo/toutiao
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
今日头条科技新闻接口爬虫
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 100.0%