I have developed multiple spiders, including one powering a broad niche (minority language) search engine with separate web and news search. This experience includes both processing the data heavily to prepare it for further use, and interfacing with complex back end systems.