8000 company scraper keeps refresh the webpage and can not find any elements · Issue #238 · joeyism/linkedin_scraper · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
company scraper keeps refresh the webpage and can not find any elements #238
Open
@yuanchenGH

Description

@yuanchenGH

`from linkedin_scraper import Company, actions
from selenium import webdriver
import time

driver = webdriver.Chrome()

email =
password =
actions.login(driver, email, password) # if email and password isnt given, it'll prompt in terminal
time.sleep(1.3)
company = Company("https://www.linkedin.com/company/snap-inc-co/about/", driver=driver, scrape=False)
time.sleep(1.5)
company.scrape()
`

It successfully logged in, opened the company page then refresh it 3 times within seconds and throw the error:

NoSuchElementException Traceback (most recent call last)
Cell In[15], line 13
11 company = Company("https://www.linkedin.com/company/snap-inc-co/about/", driver=driver, scrape=False)
12 time.sleep(1.5)
---> 13 company.scrape()

File d:\anaconda3\Lib\site-packages\linkedin_scraper\company.py:92, in Company.scrape(self, get_employees, close_on_complete)
90 def scrape(self, get_employees=True, close_on_complete=True):
91 if self.is_signed_in():
---> 92 self.scrape_logged_in(get_employees = get_employees, close_on_complete = close_on_complete)
93 else:
94 self.scrape_not_logged_in(get_employees = get_employees, close_on_complete = close_on_complete)

File d:\anaconda3\Lib\site-packages\linkedin_scraper\company.py:213, in Company.scrape_logged_in(self, get_employees, close_on_complete)
210 section_id = 3
211 #section ID is no longer needed, we are using class name now.
212 #grid = driver.find_elements_by_tag_name("section")[section_id]
--> 213 grid = driver.find_element(By.CLASS_NAME, "artdeco-card.org-page-details-module__card-spacing.artdeco-card.org-about-module__margin-bottom")
214 print(grid)
215 descWrapper = grid.find_elements(By.TAG_NAME, "p")

File d:\anaconda3\Lib\site-packages\selenium\webdriver\remote\webdriver.py:770, in WebDriver.find_element(self, by, value)
767 raise NoSuchElementException(f"Cannot locate relative element with: {by.root}")
768 return elements[0]
...
(No symbol) [0x00007FF63033315D]
(No symbol) [0x00007FF630322979]
BaseThreadInitThunk [0x00007FF8FFF17374+20]
RtlUserThreadStart [0x00007FF9016DCC91+33]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0