Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a

Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nearshore country (m/f/d)

Freelance/Contracting project Dortmund Start date: asap Reference number: 899110/1

Diesen Job teilen oder drucken

Responsibilities

• Analysis and Indexing of Appropriate Web Sources: The contractor shall identify and evaluate appropriate sources
• Design of the crawling and extraction solution:Development of a technical target architecture
• Development of Crawling, Extraction, and AI Components
• Data Structuring and Provision: Transformation of the extracted information
• Quality Assurance and Domain Validation: Implementation of procedures and a Databricks app
• Pilot Testing
• Documentation and Handover

Profile

• Web Crawling & Web Scraping: Scrapy, Beautiful Soup, or comparable frameworks, HTML, DOM, and table analysis, working with dynamic web pages and download structures
• Information Extraction & Generative AI: LLM & NLP, prompt engineering and structured output formats, classification, mapping, and duplicate detection
• Interface and Document Processing: Rest APIs, PDF and datasheet extraction, Table extraction and schema mapping
• Data Engineering & Databricks, Python, Pandas and PySpark, Delta Lake, and development of Databricks apps

Benefits

• A very renowned company
• Possibilities to work remote