Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nearshore country (m/f/d)

Analysis and Indexing of Appropriate Web Sources: The contractor shall identify and evaluate appropriate sourcesDesign of the crawling and extraction solution:Development of a technical target architectureDevelopment of Crawling, Extraction, and AI ComponentsData Structuring and Provision: Transformation of the extracted informationQuality Assurance and Domain Validation: Implementation of procedures and a Databricks app Pilot TestingDocumentation and Handover Web Crawling & Web Scraping: Scrapy, Beautiful Soup, or comparable frameworks, HTML, DOM, and table analysis, working with dynamic web pages and download structuresInformation Extraction & Generative AI: LLM & NLP, prompt engineering and structured output formats, classification, mapping, and duplicate detectionInterface and Document Processing: Rest APIs, PDF and datasheet extraction, Table extraction and schema mappingData Engineering & Databricks, Python, Pandas and PySpark, Delta Lake, and development of Databricks apps A very renowned companyPossibilities to work remote IT is and always has been our core business that laid the foundation for Hays' success. We are the biggest privately owned IT recruitment agency in Germany and offer the best jobs for every career level – whether you are interested in vacancies in agile SMEs or international DAX groups. Hays masters the entire IT job spectrum, from support to software architecture or digitalisation – thanks to our broad portfolio, we have something for everyone. In the last decades, we were able to support numerous IT experts with choosing the right path for a successful career, positioning ourselves as their lifelong partner. Our highly specialised consultants can cater to your every wish and expectation and will prepare you for interviews and contract negotiations. Give it a try and learn what the market has to offer – our services are free of charge, non-binding and discreet! We look forward to hearing from you.