Asks for

The ad says“Build AI evaluation, not just pass/fail: Use Langfuse to build eval harnesses for extraction quality, golden datasets, scoring against ground truth, and drift detection that gates on quality, not just green checks.”
The ad says“Full-stack app testing: You test real web apps end to end, UI flows, API contracts, and data integrity, and you know what makes a suite flaky and how to fix it. Strong with Playwright and comfortable in TypeScript and/or Python.”
The ad says“Own quality across the product: Full E2E and regression coverage of our Next.js frontend and backend APIs, from critical user journeys to the edge cases, so every release ships with confidence.”
The ad says“Build AI evaluation, not just pass/fail: Use Langfuse to build eval harnesses for extraction quality, golden datasets, scoring against ground truth, and drift detection that gates on quality, not just green checks.”
The ad says“We already have a Playwright/TypeScript E2E suite covering our core flows, but it needs a real owner. Your job is to own quality across the whole product: our Next.js web app, our backend APIs, and the AI document-extraction engine underneath. You'll broaden coverage across the app, make the suite trustworthy (fast, stable, and run on every change), and extend it into an AI-driven evaluation pipeline, so neither an everyday feature change nor a prompt or model change can silently break what we ship.”
The ad says“Full-stack app testing: You test real web apps end to end, UI flows, API contracts, and data integrity, and you know what makes a suite flaky and how to fix it. Strong with Playwright and comfortable in TypeScript and/or Python.”
The ad says“We already have a Playwright/TypeScript E2E suite covering our core flows, but it needs a real owner. Your job is to own quality across the whole product: our Next.js web app, our backend APIs, and the AI document-extraction engine underneath. You'll broaden coverage across the app, make the suite trustworthy (fast, stable, and run on every change), and extend it into an AI-driven evaluation pipeline, so neither an everyday feature change nor a prompt or model change can silently break what we ship.”

Your mission

We're a fast-moving, lean EU team looking for a high-agency QA engineer based in Kosovo, with zero middle management. You'll work alongside our founder and our Kosovo engineering team, report directly to the Head of Engineering, and own quality end-to-end.

We already have a Playwright/TypeScript E2E suite covering our core flows, but it needs a real owner. Your job is to own quality across the whole product: our Next.js web app, our backend APIs, and the AI document-extraction engine underneath. You'll broaden coverage across the app, make the suite trustworthy (fast, stable, and run on every change), and extend it into an AI-driven evaluation pipeline, so neither an everyday feature change nor a prompt or model change can silently break what we ship.

Note: CVs filled with LLM slop will be immediately disqualified. We want to see your actual experience in your own words.

What You'll Do

Own quality across the product: Full E2E and regression coverage of our Next.js frontend and backend APIs, from critical user journeys to the edge cases, so every release ships with confidence.
Make the suite trustworthy: Take ownership of our existing Playwright/TypeScript suite, drive down flakiness, and get it green and trusted as a CI gate the team relies on.
Harden the email → extraction → downstream pipeline: We have early coverage of our email-to-document flow. You'll turn it into the rock-solid regression backbone that protects extraction accuracy and downstream processing.
Build AI evaluation, not just pass/fail: Use Langfuse to build eval harnesses for extraction quality, golden datasets, scoring against ground truth, and drift detection that gates on quality, not just green checks.
Use AI to test faster: Lean on AI to generate cases and synthetic documents, triage failures, and review diffs, wherever it makes the suite broader and faster than hand-written tests alone.
Make testability a first-class concern: Partner with engineers so features ship testable by design, with clear, actionable failure reporting.

Your profile

Full-stack app testing: You test real web apps end to end, UI flows, API contracts, and data integrity, and you know what makes a suite flaky and how to fix it. Strong with Playwright and comfortable in TypeScript and/or Python.
3+ years in QA / test engineering: We hire experienced QA only. What matters is depth of ownership: you've built and run automation strategy and frameworks yourself, not just executed someone else's test plans. A sharp engineer who's shipped real automation beats a coaster with twice the tenure.
High Agency: You don't wait for perfectly scoped tickets. You unblock yourself, make pragmatic decisions, and take extreme ownership of quality.
AI-first instincts: You actively use AI in your testing workflow, and you're keen to take on the harder problem of testing non-deterministic AI output, or can show real aptitude for it.

Nice to Have

Experience testing non-deterministic systems: evaluating LLM/ML outputs, building eval sets, reasoning about quality metrics over pass/fail.
Data/ML pipeline testing or MLOps background.
Document processing, OCR, or data-extraction domain experience.
Familiarity with Prisma, async/queue-based systems, or complex event patterns.

Why us?

High Ownership & Influence: You'll define how Holocene does QA. Your test strategy directly protects the product and the roadmap.
Elite, Lean Team: Work with highly technical peers in an environment where experimentation is expected and red tape is nonexistent.
Fast Decisions: We respect your time. Just 2 interviews, with a final decision within 48 hours.