.Net Software developer
Hi. I have 5 years’ experience scraping development, bot automatization, Automatization register, Data Mining. Web Crawling. Data Scraping.
I can -
Reverse enginering obfuscated JavaScript for create faster scraper and used requests.
Emulating requests headers.
Switching proxy (http, https, socks 4, socks 5).
Multi-Threading work (can parallel work on some pages (5 – 10 – 50 - 500)).
use graphQl api
I use –
Languages – C#, JavaScript (Node JS)
Libraries –
Angle, HtmlAgilityPack – Parsing DOM three (its elements on web page).
RestSharp, ExtremeNet, HttpClient – Request Emulating (emulating actions of user as example click to button, going to page via catalog or menu, etc..).
Scrappy – framework for faster creating scraper - for simple site.
Entity Framework – ORM for database.
CefSharp (Chromium Embedded Framework Wrapper) – browser emulating with JS executions for hard emulated site.
Databases –
Microsoft SQL Server
PostgreSQL