<jupyter_start><jupyter_text>Web Driver Start<jupyter_code>#from webdriver_manager.chrome import ChromeDriverManager
#driver = webdriver.Firefox()
driver = webdriver.Chrome('D:\Software\chromedriver_win32\chromedriver.exe')<jupyter_output><empty_output><jupyter_text>Scrolling Infinite Times<jupyter_code>driver.get("https://www.medifee.com/health-packages-list.php")<jupyter_output><empty_output><jupyter_text>Parsing the Page<jupyter_code>#once the scroller has scrolled all the way extract the required data
res=driver.execute_script("return document.documentElement.outerHTML")
soup_1=soup(res,'lxml')
#gets the name and link
c1=soup_1.findAll("li")
c1
name=[]
link=[]
#date=[]
#description=[]
#rating=[]
 c1[14].text
c1[23].text
for r in range(14,24):
    x=c1[r].find("a")
    b=x.get("href")
    link.append("https://www.medifee.com"+b)
    name.append(c1[r].text)<jupyter_output><empty_output><jupyter_text>Conversion to Df and to csv<jupyter_code>column=["Test Name","Link"]
df=pd.DataFrame(columns=column)
df
df["Test Name"]=pd.Series(name)
df["Link"]=pd.Series(link)
#df["Description"]=pd.Series(description)
#df["Rating out of 5"]=pd.Series(rating)

df
df.to_csv("medifeehealthpackagelinks.csv")<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup