I'm going to scrap or crawl any website or directory that has data. I have been doing web scraping assignments and have effectively scraped / mined all sorts of websites including easy html to complicated javascript websites using technologies like scrapy, bs4 or selenium.
I can also write a web app that will interactive and easy to use web scrapping bot. I can save data to multiple file formats like .txt, .xlsx, .csv and databases like mysql, sqllite, graphql, mongodb and postgresql.
I have completed many web scrapping and web automation projects that involved a variety of complexities that are as follows:
Scrapy spiders (Crawler)
Proxy rotation
Fake user agent
Login
Captcha
eCommerce
Pagination
Infinite Scrolling
Automated tasks
Downloading images
I use python for web scrapping. I can deploy your crawler on cloud as well.
If you have any queries or confusions, you can text I charge nothing for honest advises.