Description:
Lead the design, development, and scaling of a self-hosted AI Agent Platform on AWS. Drive platform architecture, AI agent orchestration, security, and enterprise adoption. Blend hands-on engineering with strategic technical leadership.
At this time, Ally will not sponsor a new applicant for employment authorization for this position.
The Work Itself
- Architect and evolve a self-hosted, AWS-based AI Agent Platform using ECS/Fargate, Lambda, API Gateway, ALB, S3, KMS, CloudWatch.
- Design scalable, modular infrastructure supporting multiple agent frameworks (LangGraph, OpenAI Agent SDK, Autogen).
- Define patterns for agent deployment, orchestration, and multi-tenant scaling in containerized environments.
- Build intelligent multi-agent workflows with complex reasoning and tool orchestration.
- Develop reusable SDKs, templates, and developer tooling for faster adoption.
- Integrate agents with enterprise systems such as ServiceNow, Salesforce, Snowflake, Confluence, Okta, GRC platforms.
- Implement authentication and authorization using Okta OIDC, JWT, and fine-grained tool policies.
- Establish observability standards including logging, metrics, tracing, and governance dashboards.
- Ensure compliance with enterprise AI governance and risk management frameworks.
- Mentor senior engineers and act as technical thought leader across teams.
- Collaborate with cross-functional stakeholders to drive enterprise-wide platform adoption.
Skills
The Skills You Bring
- Bachelor’s or Master’s in Computer Science, AI/ML, Data Science, or related experience preferred
- 5+ years of software engineering experience required
- 3+ years in senior or principal roles preferred
- Proven experience architecting and scaling distributed systems and AI platforms on AWS.
- Strong Python skills and familiarity with async frameworks like FastAPI and asyncio.
- Experience with AI agent frameworks (LangGraph, Autogen, OpenAI SDK, MCP).
- Knowledge of LLM orchestration, vector databases, retrieval augmentation, prompt patterns required
- Expertise in container orchestration, serverless architectures, and CI/CD pipelines (GitLab, Terraform) preferred.
- Familiarity with cloud security, IAM, networking, and enterprise authentication (Okta) preferred.
- Excellent communication skills and ability to influence technical and executive stakeholders.
- Strategic thinker with hands-on problem-solving skills.
- Proven record of building platforms or frameworks adopted at scal