The ad says“Run short, instrumented spikes and benchmarking on new models, tools and frameworks: LLMs, agentic systems, vector databases, orchestration frameworks, copilots, assistants and more”
The ad says“Build automation and tooling to measure quality, robustness, latency and cost, including regression tracking over time”
The ad says“Strong knowledge of cloud infrastructure, preferably AWS, and the ability to run experimental workloads securely and cost-consciously”
The ad says“Working knowledge of LLM-as-judge methods and their failure modes, alongside human evaluation, inter-annotator agreement, and a view on when each is appropriate”
The ad says“Run short, instrumented spikes and benchmarking on new models, tools and frameworks: LLMs, agentic systems, vector databases, orchestration frameworks, copilots, assistants and more”
The ad says“Run short, instrumented spikes and benchmarking on new models, tools and frameworks: LLMs, agentic systems, vector databases, orchestration frameworks, copilots, assistants and more”
The ad says“Strong software engineering fundamentals and the ability to rapidly build high-quality experimental systems”
The ad says“Able to reason about statistical significance on small samples, and to state confidence honestly rather than over-reading a result”
The ad says“Design and maintain evaluation environments (e.g. datasets, prompts, scenarios, telemetry) to test models under realistic constraints”
The ad says“Run short, instrumented spikes and benchmarking on new models, tools and frameworks: LLMs, agentic systems, vector databases, orchestration frameworks, copilots, assistants and more”
You will join the AI Lab, a team whose mission is to validate high-value emerging AI and automation technologies and de-risk their adoption across dLocal
This is a rare opportunity to work at the frontier of applied AI in fintech: running rigorous experiments on the latest models and tools, and turning results into decisions that shape how a global payments company adopts AI
This is a senior individual-contributor role
It does not require direct people management, but it carries significant technical influence within the Lab and across the teams that consume its work
As a Staff AI Engineer in the AI Lab, you own technology scouting, prototyping, and evaluation for dLocal
You will run instrumented spikes and benchmarking on emerging AI technologies, produce clear recommendations for stakeholders across engineering, business, legal and IT based on your prototypes, and coordinate hand-offs to the teams that take validated technologies into production
For promising technologies, you will help define the patterns, guardrails, and technical requirements needed for adoption and coordinate the hand-off to the engineering teams responsible for productionizing them
Technology Scouting & Evaluation
Run short, instrumented spikes and benchmarking on new models, tools and frameworks: LLMs, agentic systems, vector databases, orchestration frameworks, copilots, assistants and more
Compare vendor and open-source options, documenting trade-offs across quality, cost, latency, security and integration complexity
Deliver concise decision memos with clear recommendations: adopt, watch, or avoid
Evaluation Harnesses & Sandboxes
Design and maintain evaluation environments (e.g. datasets, prompts, scenarios, telemetry) to test models under realistic constraints
Build automation and tooling to measure quality, robustness, latency and cost, including regression tracking over time
Ensure every evaluated technology has benchmark coverage and a documented risk and limitations view
Prototyping & Technical Validation
Build enough of a system to understand how a technology behaves under realistic conditions, not just in vendor demos or isolated examples
Explore architecture, integration patterns, operational constraints, security boundaries, and failure modes through working prototypes
Determine what must be true for a proof of concept to become a viable production capability
Prefer focused prototypes that answer specific technical questions over prematurely building production systems
Recommendations, Readiness & Hand-offs
Translate technical findings into clear decision memos for both technical and non-technical stakeholders
For validated technologies, produce readiness guidance covering recommended patterns, guardrails, known limitations, operational considerations, and integration requirements
Coordinate hand-offs to the engineering teams responsible for productionization
Support those teams during the transition when deep context from the evaluation is required, without becoming the permanent owner of the resulting system
Track what happens after Lab recommendations and use those outcomes to improve future evaluation methods
Governance, Risk & Standards
Work with Security, Legal, Compliance and other AI teams to document risk assessments, mitigations and governance recommendations for each evaluated technology
Maintain checklists, decision templates and lightweight standards reusable across evaluations and by partner teams
Incorporate learnings from third-party AI tooling already in use, such as external copilots and the AWS AI suite, into adoption guidelines
Collaboration, Mentoring & Community
Partner with other AI teams and domain teams to ensure clear boundaries and smooth collaboration
Participate in hiring as a technical evaluator and culture champion
Mentor engineers in the Lab and adjacent teams on evaluation methods, benchmarking and experimental design
Share knowledge through internal write-ups, tech talks and occasional external meetups and conferences
Preferred experience
Experience building agentic or multi-step AI systems involving tool use, orchestration, state, retrieval, or external integrations
8+ years of software engineering experience, including significant experience operating at senior or Staff-level scope
Experience with observability, telemetry, testing, and benchmarking of complex systems
Ability to reason about system architecture, reliability, scalability, asynchronous workflows, and distributed components where relevant
Deep hands-on experience building and evaluating systems based on LLMs and modern AI tooling
Strong knowledge of cloud infrastructure, preferably AWS, and the ability to run experimental workloads securely and cost-consciously
Track record of designing experiments or benchmarks that influenced meaningful technical decisions
Strong software engineering fundamentals and the ability to rapidly build high-quality experimental systems
Experience constructing evaluation datasets: task selection, labelling, holdout discipline, and keeping a set useful as models improve
Working knowledge of LLM-as-judge methods and their failure modes, alongside human evaluation, inter-annotator agreement, and a view on when each is appropriate
Familiarity with regression tracking, telemetry and versioning, so that a result stays reproducible months later
Able to reason about statistical significance on small samples, and to state confidence honestly rather than over-reading a result
Track record designing and running benchmarks that compare AI models and tools under real constraints
Experience writing short, opinionated decision memos that help others move fast
Comfortable making trade-off calls across quality, latency, cost and vendor lock-in, and documenting them clearly
Able to turn ambiguous “we should try this new thing” ideas into well-scoped evaluation plans with clear hypotheses and metrics
Can explain technical results to non-specialists in concrete, concise terms
Able to influence without authority, aligning teams around shared standards and guardrails
Experience working with platform, product and operations teams to align evaluations with real use cases
Curious and biased toward experimentation, combined with disciplined measurement and risk awareness
Comfortable in a small, high-leverage team without embedded PMs. You structure your own work and keep stakeholders informed
Builder attitude: you prefer reusable tools, templates and playbooks over one-off work
Recruitment process
Our Talent Acquisition team is invested in creating the best candidate experience possible, so don’t worry, you will definitely hear from us. We will review your CV and keep you posted by email at every step of the process!
