Scrapy tutorials for the master tag data science spring 2021 edition.
This Scrapy project allows to scrape data of the following websites:
Install the Scrapy library in your environment
# PyPI
pip install scrapy# conda
conda install -c conda-forge scrapyTo make sure you have installed the library correctly, run the scrapy command on the shell.
Scrape the data and save the results to json data
spider_name: [books, quotes]filename: any value
scrapy crawl <spider_name> -o data/<filename>.json