Data Engineer | Data Scientist — Unstructured Data & AI Pipelines

About the role

Neurons Lab is looking for a part‑time Data Engineer / Data Scientist to build unstructured‑first data pipelines for a European private investment group. The project involves capturing, normalising and enriching communications, documents and historical archives to create a context layer that powers AI agents.

Key responsibilities

• Implement capture pipelines for calls, emails, Slack, messengers with speaker attribution and opt‑out controls.
• Back‑fill years of historical communications, parse PDFs, scanned decks, spreadsheets and deduplicate records.
• Develop entity‑resolution and record‑linkage across multiple identifiers (Slack handles, email aliases, calendar invites).
• Build chunking, embedding and vector‑graph storage using pgvector, OpenSearch or Pinecone‑class solutions.
• Design incremental sync mechanisms, handle edits, deletions and ensure cost‑effective scaling.
• Apply PII detection, redaction, retention policies and provide audit evidence for security teams.
• Orchestrate pipelines with Airflow or Step Functions, add monitoring and alerting.
• Document runbooks for client hand‑over and ongoing operations.

Required profile

• 4+ years of data‑engineering experience with unstructured or semi‑structured data.
• Proven ability to integrate multiple third‑party APIs and perform historical backfills.
• Experience delivering pipelines that feed LLM‑based retrieval systems.
• Comfortable working as the sole data engineer in a small, distributed pod.
• Familiarity with GDPR, data residency and security expectations in financial services.

Required skills

• Python
• SQL
• Airflow or equivalent orchestration tool
• AWS and/or GCP data services
• Vector stores (pgvector, OpenSearch, Pinecone)
• Graph store integration
• Google Workspace, Microsoft 365, Slack, CRM API integration
• Entity resolution / record linkage techniques
• PII detection, redaction and retention handling