A full-stack application that scrapes Amazon pages using their ASIN number, saves relevant information to a database and displays the information in a table.
- Ensure you have ruby 2.5.0 installed on your machine and postgresql
- Ensure you have chromedriver installed locally (if you are using brew, use the command
brew cask install chromedriver) - Clone this repo to your machine
- run
bundle installvia command line - run
rails svia command line - visit
http://localhost:3000/products
- Run
rspecon the command line
- Utilizes Waitr gem to overcome the delayed loading amazon page
- Nokogiri is used for parsing page contents
- Custom service implemented for scraping exact nodes for Nokogiri using xpath and css lookups
- Uses VCR to record and real http responses. This is useful for ensuring that the application appropriately handles a variety of page layouts and that all parsing is working as it should.