G
GlassDollarvia muse
Senior Data Engineer (m/f/d)
Berlin, BE, DEPosted 5d ago
Data EngineerSeniorFull-time
About the Role
#### **TL;DR**
You build and run the pipelines behind GlassDollar's company data: ingestion, scraping, enrichment, entity resolution, LLM extraction and signal detection – from first prototype to reliable production. You sit in the product team, report to our Product Owner and work closely with our engineers.
Our data has grown faster than our engineering around it. Feeds run, but nothing tells us when they stop. Most relationship records aren't classified yet, duplicates slip through, and some signals users ask for, like headcount changes, have no reliable live source. Making this robust, observable and fast is the job.
**Who uses your work**
* **People in vclOS:** Search, lists, projects, PoC tracking, recommendations and weekly company updates.
* **Our AI agents \& search:** Clean fields with provenance, confidence and freshness, measured on our search benchmark.
* **API and MCP partners:** Stable schemas and fields to build on.
* **Marketing:** Directory, rankings and reports through a quarterly data release.
When these conflict, the product wins.#### **What we're looking for**
**What you own**
* **Ingestion and pipelines:** vendor feeds, APIs and scraping jobs – scheduled, idempotent, monitored, with backfills and alerts when a feed stops or drifts.
* **Schema and data model:** the Postgres schema and data dictionary as the source of truth, including migrations and the Hasura layer on top.
* **Entity resolution:** matching and deduplicating companies across sources, with measurable precision and recall.
* **LLM extraction and classification:** pipelines that turn unstructured sources into structured records, evaluated on labelled samples and tuned for cost per record.
* **Signals:** funding, acquisitions, shutdowns, case studies and headcount as event pipelines with change detection and correct event dates.
* **Data quality and observability:** automated tests and metrics for coverage, accuracy, freshness and completeness.
* **Data sources:** you test vendors, APIs and scraping partners on real samples and give a clear recommendation on quality and cost per datapoint.
You join sprint planning and review, triage data errors reported by users and agents, and take datapoints from prototype to production yourself.#### **What you'll do**
**Your first 90 days**
By day 90: every feed is monitored and alerting, relationship data is classified with measured precision, entity resolution has a baseline, 3–5 new datapoints are live in production, and our search benchmark has moved against a target you set in month 1\.
**Month 1: baseline and observability**
* Inventory every pipeline, vendor feed and scraping job; give each one an owner, runbook, monitoring and alerting.
* Data dictionary v1 for the top 25 datapoints, with source, lineage and quality numbers.
* Automated quality checks (coverage, freshness, nulls, duplicates) running on every load.
* Quick win: Scale partnership data point collection.
**Month 2: corporate–technology relationships**
* Model relationship types in the schema: mention, partnership, PoC, implementation, investment, acquisition.
* Build an LLM classification pipeline for existing and incoming records; report precision on a labelled sample and cost per record.
* Set a recall and lag baseline, then improve the relationship pipeline against it.
* Integrate a live acquisition source and keep acquired companies out of recommendations.
**Month 3: new datapoints, entity resolution and agents**
* Run 6–8 time\-boxed datapoint experiments, each ending in ship, kill or park. Ship 3–5 to production.
* Entity resolution v2: measurable matching across sources, with a review queue for uncertain cases.
* Add provenance, confidence and freshness to the top 25 fields, and make customer data (lists, PoCs, notes) usable by agents with proper permissions.
* Document the data platform: architecture, runbooks and on\-call handover.
#### **Requirements**
**Who you are**
A s
You build and run the pipelines behind GlassDollar's company data: ingestion, scraping, enrichment, entity resolution, LLM extraction and signal detection – from first prototype to reliable production. You sit in the product team, report to our Product Owner and work closely with our engineers.
Our data has grown faster than our engineering around it. Feeds run, but nothing tells us when they stop. Most relationship records aren't classified yet, duplicates slip through, and some signals users ask for, like headcount changes, have no reliable live source. Making this robust, observable and fast is the job.
**Who uses your work**
* **People in vclOS:** Search, lists, projects, PoC tracking, recommendations and weekly company updates.
* **Our AI agents \& search:** Clean fields with provenance, confidence and freshness, measured on our search benchmark.
* **API and MCP partners:** Stable schemas and fields to build on.
* **Marketing:** Directory, rankings and reports through a quarterly data release.
When these conflict, the product wins.#### **What we're looking for**
**What you own**
* **Ingestion and pipelines:** vendor feeds, APIs and scraping jobs – scheduled, idempotent, monitored, with backfills and alerts when a feed stops or drifts.
* **Schema and data model:** the Postgres schema and data dictionary as the source of truth, including migrations and the Hasura layer on top.
* **Entity resolution:** matching and deduplicating companies across sources, with measurable precision and recall.
* **LLM extraction and classification:** pipelines that turn unstructured sources into structured records, evaluated on labelled samples and tuned for cost per record.
* **Signals:** funding, acquisitions, shutdowns, case studies and headcount as event pipelines with change detection and correct event dates.
* **Data quality and observability:** automated tests and metrics for coverage, accuracy, freshness and completeness.
* **Data sources:** you test vendors, APIs and scraping partners on real samples and give a clear recommendation on quality and cost per datapoint.
You join sprint planning and review, triage data errors reported by users and agents, and take datapoints from prototype to production yourself.#### **What you'll do**
**Your first 90 days**
By day 90: every feed is monitored and alerting, relationship data is classified with measured precision, entity resolution has a baseline, 3–5 new datapoints are live in production, and our search benchmark has moved against a target you set in month 1\.
**Month 1: baseline and observability**
* Inventory every pipeline, vendor feed and scraping job; give each one an owner, runbook, monitoring and alerting.
* Data dictionary v1 for the top 25 datapoints, with source, lineage and quality numbers.
* Automated quality checks (coverage, freshness, nulls, duplicates) running on every load.
* Quick win: Scale partnership data point collection.
**Month 2: corporate–technology relationships**
* Model relationship types in the schema: mention, partnership, PoC, implementation, investment, acquisition.
* Build an LLM classification pipeline for existing and incoming records; report precision on a labelled sample and cost per record.
* Set a recall and lag baseline, then improve the relationship pipeline against it.
* Integrate a live acquisition source and keep acquired companies out of recommendations.
**Month 3: new datapoints, entity resolution and agents**
* Run 6–8 time\-boxed datapoint experiments, each ending in ship, kill or park. Ship 3–5 to production.
* Entity resolution v2: measurable matching across sources, with a review queue for uncertain cases.
* Add provenance, confidence and freshness to the top 25 fields, and make customer data (lists, PoCs, notes) usable by agents with proper permissions.
* Document the data platform: architecture, runbooks and on\-call handover.
#### **Requirements**
**Who you are**
A s