AI Engineer (LLM/RAG) (m/w/d)
Nejo · Cologne
What this role requires
4 requirements, read out of the advert rather than guessed from the job title:
Also mentioned, not required: Docker, Azure, Anthropic API, OpenAI API, Vercel AI SDK, English (B2), German. Worth having, but their absence is not what gets a CV filtered out.
See how often each of these is required across open backend roles in Europe.
Free, no card. Tells you which of them your CV evidences and which it only implies.
Job description
For a company in the fast-growing AI implementation market we are looking for an experienced AI Engineer, starting immediately. The company operates LLM-based systems in production: content generation pipelines, retrieval-augmented generation (RAG) over internal documents, and automated workflows deeply integrated with their business systems. The stack is TypeScript and Next.js end to end.
These systems are already live. As AI Engineer you take ownership of them, improve their reliability and quality, and extend them to new use cases. The role combines applied LLM engineering with solid backend engineering in TypeScript. It does not involve training or fine-tuning foundation models.
Stack: TypeScript, Next.js, Node.js, Postgres with pgvector, Docker, Azure, Anthropic and OpenAI APIs, Vercel AI SDK.
Tasks
Take over and maintain the existing LLM pipelines: assess the current architecture, identify failure modes, prioritise fixes, and refactor and extend without disrupting production
Own the RAG systems end to end: document ingestion and parsing, chunking, indexing, hybrid retrieval (BM25 and vector), query rewriting, reranking, grounded generation with citations
Read the full description (40 more sections)Show less
Implement and maintain chunk-level access control, index freshness and tenant isolation across retrieval systems
Develop content generation pipelines that deliver consistent quality at volume, including human review steps
Build and operate automated workflows against internal and third-party business systems (ERP, CRM, email, internal APIs), with durable and idempotent execution, retry and dead-letter handling, and approval steps for irreversible actions
Establish an evaluation framework for systems currently running without one: golden datasets derived from observed production failures, retrieval metrics and more
Implement observability across the full request path
Optimise cost and latency through prompt caching, batching, model routing and use of smaller models where appropriate
Assess where deterministic logic is the better solution and implement it accordingly
Work directly with non-technical colleagues to specify and validate automated processes
Requirements
Professional experience with at least one LLM-based system in production use, including responsibility for its operation and incident handling
TypeScript and Node.js at an advanced level: strict typing of non-deterministic model output, async and concurrency patterns, streaming responses, structured error handling
Next.js in production: App Router, route handlers, server actions, streaming to the client
Practical retrieval expertise: hybrid search, embedding model selection, cross-encoder reranking, metadata filtering, permission-aware retrieval, and structured diagnosis of poor retrieval quality
Experience processing real-world documents: PDFs with tables, scanned material, DOCX, HTML, including layout-aware parsing, OCR and evidence-based chunking
Structured outputs and tool calling as part of your everyday work: JSON Schema, Zod or comparable runtime validation, function calling, handling of malformed or partial output, context window management
Designed and run LLM evaluations
Experience with LLM tracing and evaluation tooling in a TypeScript codebase (e.g. Braintrust, Langfuse, Promptfoo, OpenTelemetry or Arize Phoenix)
Familiar with Postgres including vector search (pgvector or a comparable vector store), Docker, Git, CI/CD and one major cloud platform
Working experience with the Anthropic and/or OpenAI TypeScript SDKs
Confident communication in English, German is a plus
If you have experience with any of the following, that's a plus:
durable workflow execution for long-running, unattended processes (Temporal, Inngest or comparable)
agent orchestration in production, tool calling, recovery, multi-step workflows (Vercel AI SDK, LangGraph, Mastra, Claude Agent SDK, MCP TypeScript SDK)
integration experience with enterprise systems such as ERP or CRM platforms like SAP
security and data protection in LLM systems, prompt injection and data exfiltration defences, PII handling, GDPR-compliant design, EU-hosted or self-hosted inference
structured or graph-based retrieval for entity-heavy data
experience migrating live pipelines to a new model, embedding model or index without quality regression
Azure DevOps, Pipelines, Repos and Boards
Benefits
Hybrid setup, 2 office days per week in the light-flooded office in the Belgian Quarter in Cologne (with two balconies and the best view over the city ;))
AI implementation, the growth market of the coming years
Trust-based working hours with overtime compensation
Plenty of creative freedom and room for your own ideas
Continuous development, professional, strategic and technological
An open feedback culture, transparent communication and short decision paths
Regular team events and workations
Salary from €60,000 gross per year, ofc with room upwards depending on your experience
The company is an international team and actively fosters an inclusive environment. Applications from all genders and identities are welcome, regardless of origin, age, religion, sexual orientation or disability. Women are still underrepresented in the tech industry and are explicitly encouraged to apply.
We look forward to your application!
Find Jobs in Germany on Arbeitnow
You will apply. Then you will hear nothing.
And no one will tell you what was wrong. See it before you send: your ATS score, every weak line, and the fix for each.
- 1
Drop in your CV
One PDF, thirty seconds. No card.
- 2
See what is wrong with it
Every weak passage, quoted from your own CV, with the line to replace it.
- 3
Apply where you fit
Every European role ranked against what your CV actually says.
- Free, no card
- Your CV file is deleted after parsing
- Refreshed every 6 hours