Lead AI Developer
About the job
Job Title: Lead Backend Engineer LLM
Location: Greece - Remote
Department: Engineering
*Job Overview *
HealthBook+ is a startup company with a need for a Lead Backend Engineer LLM-Python. As a Senior Backend Software Engineer on our LLM engineering team, you’ll be at the forefront of integrating AI into healthcare delivery. Your tasks will include designing and implementing Python-based services that leverage Large Language Models to enhance our core healthcare platform. Working closely with our distributed engineering and product teams,
*Key Responsibilities: *
- Lead LLM team for aspects of development
- Architect and build scalable microservices that power our LLM-enabled features
- Mentoring lower team members
- Collaborate with other Software, ML and Infrastructure engineers to optimize LLM integration and deployment
- Design and implement LLM orchestration services, including prompt management, model switching, and response streaming
- Build robust evaluation pipelines to measure and improve LLM output quality and consistency
- Develop scalable APIs for LLM-powered features, including context injection and retrieval augmentation
- Implement efficient caching and optimization strategies for LLM inference
- Create monitoring systems for tracking token usage, latency, and other LLM-specific metrics
- Work on prompt engineering and chain-of-thought implementations
- Ensure compliance with healthcare regulations while working with LLM outputs
- Participate in technical design discussions and code reviews
- Monitor and maintain services in production, including LLM Observability using tools like LangFuse
*Requirements *
- Strong proficiency in Python and experience building production-grade backend services
- Understanding of prompt engineering principles and LLM evaluation metrics
- Experience with vector databases (like Pinecone, Weaviate, or similar) for semantic search
- Familiarity with streaming architectures for real-time LLM responses
- Experience implementing rate limiting and failover strategies for API services
- Strong grasp of software engineering best practices, including testing, documentation, and version control
- Excellent problem-solving skills and attention to detail
- Strong written and verbal communication skills
- Experience with LLM monitoring tool LangFuse.
- Familiarity with SAFe Agile development process
*You'd Be a Great Fit If You Have *
- Experience fine-tuning or implementing RAG (Retrieval Augmented Generation) systems
- Experience with LLM frameworks such as LangGraph, LangChain, LlamaIndex, or similar orchestration tools
- Familiarity with different LLM providers (OpenAI, Anthropic, etc.) and their APIs
- Familiarity with the use of Open LLMs, either self-hosted or on through a third party provider, e.g. Amazon Bedrock
- Knowledge of LLM output validation and safety measures
- Experience with embeddings and semantic search implementations
- Background in prompt engineering or LLM evaluation metrics
- DevOps experience, particularly with containerized service deployment, GPU-enabled compute environments and scaling
- Knowledge of US and EU healthcare data privacy regulations and security best practices
- Experience working in distributed teams across multiple time zones
- Familiarity with US Healthcare
*Location & Work Style *
- All-remote position
- Flexible working hours
- Occasional late meetings to collaborate with US-based team members