Product Analyst

Description:

The Product Analyst (Context Designer) is responsible for designing, documenting, and optimizing AI-powered conversational experiences that support business operations and customer interactions. This role works at the intersection of product management, business analysis, and emerging AI technologies, translating complex business requirements into conversational flows, prompts, knowledge sources, and evaluation scenarios. The successful candidate will collaborate with cross-functional teams to improve AI-driven customer experiences across sales, servicing, fulfillment, and production workflows. This position is analytical and product-focused rather than software engineering, requiring strong communication, documentation, and business process analysis skills.

Qualifications

Required:

• • Write, refine, and configure natural-language prompts that guide voice AI agents through defined business processes and customer interactions.
  • Design conversational flows, decision paths, response logic, and escalation scenarios based on approved business requirements.
  • Partner closely with sales, servicing, fulfillment, production, and operational teams to capture and document AI use cases.
  • Facilitate requirements discussions to clarify business rules, negotiate priorities, identify exceptions, and resolve process gaps.
  • Translate business objectives into detailed functional requirements, user stories, acceptance criteria, and conversational design specifications.
  • Review, validate, organize, and curate operational documentation, including PDFs, procedures, policies, and knowledge articles.
  • Help establish and maintain verified source-of-truth content used by retrieval-augmented generation and vector database solutions.
  • Plan and execute A/B tests across prompts, conversational approaches, and response configurations.
  • Analyze AI interaction data to identify opportunities to improve accuracy, tone, completion rates, customer experience, and issue resolution.
  • Support the Product Owner in developing and documenting a "Golden Dataset" of representative test scenarios for automated and manual AI evaluations.
  • Collaborate with platform engineers and technical teams to validate integrations, API interactions, data exchanges, and end-to-end platform functionality.
  • Identify inconsistencies, missing information, edge cases, and potential risks within AI-generated responses and conversational workflows.
  • Document findings, testing results, recommendations, and configuration changes in a clear and repeatable manner.
  • Stay current with emerging generative AI, conversational AI, prompt-design, and retrieval-augmented generation practices.
    • • 3-5 years of experience as a Product Analyst, Business Analyst, Product Operations Analyst, or in a comparable analytical role.
      • Working knowledge of generative AI concepts, including large language models, prompt design, retrieval-augmented generation, vector databases, and AI evaluation methods.
      • Strong experience gathering, documenting, validating, and negotiating business and functional requirements.
      • Demonstrated ability to translate complex business processes into clear workflows, decision logic, user stories, and acceptance criteria.
      • Strong written communication skills with the ability to create precise, natural, and contextually appropriate language.
      • Experience designing or documenting conversational workflows, customer journeys, process flows, or decision trees.
      • Strong analytical skills with experience evaluating test results, identifying patterns, and making data-informed recommendations.
      • Exceptional attention to detail, particularly when reviewing business rules, source documentation, prompts, and AI-generated outputs.
      • Ability to work effectively with product owners, business stakeholders, subject-matter experts, and engineering teams.
      • Ability to quickly learn technical concepts and communicate effectively between business and technology groups.
        • • Prior experience with Mortgage, Banking or Financial services.
          • Experience supporting conversational AI, voice AI, virtual agents, chatbots, or other prompt-driven applications.
          • Familiarity with A/B testing, AI quality measurement, model evaluation, or test-dataset development.
          • Exposure to AWS-based platforms, API integrations, or cloud-hosted AI solutions.
          • Experience working within financial services, mortgage, lending, servicing, or another highly regulated environment.
          • Familiarity with Agile product development methodologies and tools used to manage requirements, backlogs, and testing.