Simulation and Test Engineer (Conversational AI) - US Based at Andromeda Robotics

About UsAndromeda Robotics is an ambitious social robotics company with offices in Melbourne and San Francisco, dedicated to creating robots that seamlessly and intelligently interact with the human world. Our first robot, Abi, is a testament to this vision—a custom-built platform designed from the ground up to be a helpful aid and intuitive partner in aged care homes. We are a passionate, collaborative team of engineers who solve some of the most challenging problems in AI and robotics. To accelerate our development and ensure Abi's reliability, we are seeking a foundational member to build out our capabilities to train and test our robot in simulation. Our ValuesDeeply empathetic - Kindness and compassion are at the heart of everything we do.Purposely playful - Play sharpens focus. It keeps us curious, fast and obsessed with the craft.Relentlessly striving - With relentless ambition, an action bias and constant curiosity, we don’t settle.Strong when it counts - Tenacious under pressure, we expect problems and stay in motion to adapt and progress.United in action - Different minds. Shared mission. No passengers.The RoleWe are looking for a creative and driven Simulation and Test Engineer to build Andromeda's testing infrastructure for our conversational AI systems and embodied character behaviours. Your immediate focus will be creating robust test systems for Abi's voice-to-voice chatbot, social awareness perception, and gesture motor control. As this infrastructure matures, you'll extend it into simulation environments for generating synthetic training data for character animation and gesture models.The TeamYou'll work at the intersection of our character software, robotics, perception, conversational AI, controls, and audio engineering teams. We bring deep expertise from autonomous vehicles and robotics, including simulation backgrounds. You'll collaborate with product owners and technical specialists to define requirements, integrate systems, and ensure quality across our AI/ML stack. Phase 1: Build The Test FoundationDefine and stand up synthetic test environments for our AI/ML conversational stackConversational AI testing: voice-to-voice chat quality, response appropriateness, tool calling accuracyMemory system testing: context retention, recall accuracy, conversation coherenceAudio modelling and testing: multi-speaker scenarios, room acoustics, voice activity detectionPerception system testing: social awareness (face detection, gaze tracking, person tracking)Gesture appropriateness testing: Working with our Controls/ML team, create test infrastructure to validate that Abi's body gesturesCI/CD and automated regression testing for all AI/ML subsystemsCustodian of quality metrics: if they don't exist, work with stakeholders to elicit use cases, derive requirements, and establish measurable quality metricsRequirements formalisation: you're skilled at gathering, documenting, and tracing requirements back to test casesPhase 2: Scale To ML Training InfrastructureOur approach to gesture generation requires high-fidelity synthetic interaction data at scale. You'll investigate and build the infrastructure to generate this data, working closely with our character software team to define requirements and validate approaches.Extend test environments into training data generation pipelinesInvestigate and stand up simulation tools (e.g. Unity, Unreal Engine, Isaac Sim) to support our machine learning pipeline with synthetic data and validation infrastructureBuild infrastructure for fine-tuning character animation models on simulated multi-actor scenariosEnable ML-generated gesture development to augment hand-crafted animation workflowsCreate virtual environments with diverse social interaction scenarios for training and evaluationSuccess In This Role Looks LikeMonths 1-3, stabilise our conversational system with automated regression tests and measurable quality benchmarks. By month 6, deliver an integrated simulation environment enabling rapid testing and iteration across our AI/ML stack.You'll design tests that push our systems beyond their limits and find what's brittle. Through trade studies and make-vs-buy decisions, you'll establish the infrastructure, set up automatic regression tests, and trace test cases back to high-level requirements. You'll be the final guardian, verifying that our AI and machine learning systems work as intended before integration with Abi's physical platform. Your work will directly impact the speed and quality of our development, ensuring that every software build is robust, reliable, and safe.Key ResponsibilitiesArchitect and Build: Design, develop, and maintain scalable test infrastructure for conversational AI, perception, and gesture control systemsOwn Testing Pipeline: Develop a robust CI/CD pipeline for automated regression testing, enabling rapid iteration and guaranteeing quality before deploymentDevelop Test Scenarios: Create diverse audio environments, multi-actor social scenarios, and edge cases to rigorously test Abi's conversational and social capabilitiesModel with Fidelity: Implement accurate models of Abi's hardware stack (cameras, microphone array, upper body motion) as needed for test and simulation scenariosEnable Future ML Training: Design test infrastructure with an eye towards evolution into a simulation platform for generating synthetic training data for character animation and gesture modelsIntegrate and Collaborate: Work closely with the robotics, AI, and software teams to seamlessly integrate their stacks into the test infrastructure and define testing requirementsAnalyse and Improve: Develop metrics, tools, and dashboards to analyse test data, identify bugs, track performance, and provide actionable feedback to the engineering teamsIdeally You HaveBachelor or Masters in Computer Science, Robotics, Engineering, or a related field5+ years of professional experience testing complex AI/ML systems (conversational AI, perception systems, or embodied AI)Strong programming proficiency in Python (essential); C++ experience valuableHands-on experience with LLM testing, voice AI systems, or chatbot evaluation frameworksUnderstanding of audio processing, speech recognition, and/or computer vision fundamentalsExperience with testing frameworks and CI/CD tools (pytest, Jenkins, GitHub Actions, etc.)Familiarity with ML evaluation metrics and experimental designA proactive, first-principles thinker who is excited by the prospect of owning a critical system at an early-stage startupBonus PointsExperience with simulation platforms (e.g. Unity, Unreal Engine, NVIDIA Isaac Sim, Gazebo) and physics enginesExperience with character animation systems, motion capture data, or gesture generationKnowledge of reinforcement learning, imitation learning, or synthetic data generation for training ML modelsExperience with 3D modelling tools and game engine content creationUnderstanding of ROS2 for robotics integrationKnowledge of sensor modelling techniques for cameras and audioExperience building and managing large-scale, cloud-based simulation infrastructurePhD in a relevant field The expected base salary range for this role, when performed in our San Francisco office, is $150,000 - $250,000 USD, depending on skills and experience. The salary for this position may vary depending on factors such as job-related knowledge, skills, and experience. The total compensation package may also include additional benefits or components based on the specific role. Details will be provided if an employment offer is made.If you’re excited about this role but don’t meet every requirement, that’s okay—we encourage you to apply. At Andromeda Robotics, we celebrate diversity and are committed to creating an inclusive environment for all employees. Let’s build the future together.We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.