Sezzle - Principal Infrastructure Engineer
Requirements
• Strong systems fundamentals: Linux, networking, DNS, TLS, storage, concurrency, and distributed system failure modes, with the ability to debug problems across infrastructure and application boundaries. • Strong systems fundamentals • Experience operating a 24/7, high-availability platform where downtime has direct customer or revenue impact, including hands-on incident response and postmortem remediation. • 24/7, high-availability platform • Willingness to participate in an on-call rotation and demonstrated ability to recover production systems under pressure using evidence-based triage, safe mitigation, and clear technical communication. • Willingness to participate in an on-call rotation • Experience implementing and testing disaster recovery against defined recovery objectives, including restoring data and validating service recovery. • disaster recovery • Practical experience with observability, load testing, capacity planning, and safe CI/CD practices for shared production infrastructure. • observability, load testing, capacity planning, and safe CI/CD practices • Active use of AI tooling in engineering or operations, with practical judgment about its limitations and how to verify generated code, recommendations, and operational actions. • Active use of AI tooling in engineering or operations • Ability to carry ambiguous technical problems from investigation through production delivery and collaborate across engineering disciplines to resolve system-wide constraints. • Experience in fintech, payments, or banking, operating infrastructure with demanding reliability, security, and audit requirements. • fintech, payments, or banking • Experience with multi-region architectures, chaos engineering, and failure testing, including the consistency and recovery tradeoffs of distributed data systems. • multi-region architectures, chaos engineering, and failure testing • Proficiency with Prometheus, Grafana, Loki, Tempo, or comparable observability systems. • Prometheus, Grafana, Loki, Tempo • Experience building internal platform capabilities and self-service tooling, including deployment automation, progressive delivery, and reusable infrastructure components. • internal platform capabilities and self-service tooling • Experience building AI-assisted incident investigation or operational automation with restricted access, auditable execution, and clear human review points • AI-assisted incident investigation or operational automation • You have relentlessly high standards - many people may think your standards are unreasonably high. You are continually raising the bar and driving those around you to deliver great results. You make sure that defects do not get sent down the line and that problems are fixed so they stay fixed. • You’re not bound by convention - your success—and much of the fun—lies in developing new ways to do things • You’re not bound by convention • You need action - speed matters in business. Many decisions and actions are reversible and do not need extensive study. We value calculated risk-taking. • You need action • You earn trust - you listen attentively, speak candidly, and treat others respectfully. • You earn trust • You have backbone; disagree, then commit - you can respectfully challenge decisions when you disagree, even when doing so is uncomfortable or exhausting. You have conviction and are tenacious. You do not compromise for the sake of social cohesion. Once a decision is determined, you commit wholly. • You have backbone; disagree, then commit • You deliver results - you focus on the key inputs and deliver them with the right quality and in a timely fashion. Despite setbacks, you rise to the occasion and never settle. • You deliver results • You go deep: you investigate how systems behave under load and failure, follow the evidence, and fix underlying causes. • You go deep • You build and ship: you turn architecture into working code, tested infrastructure, and safe production changes. • You build and ship • You measure results: you can explain the impact of your work in availability, latency, capacity, recovery time, cost, or hours of toil removed. • You measure results • You value simplicity: you choose systems that are understandable, operable, and appropriate to the problem, and reduce unnecessary complexity. • You value simplicity • You earn trust: you listen carefully, communicate clearly, challenge technical decisions respectfully, and follow through on commitments. • You earn trust • You stay accountable for the outcome: you verify that changes work in production and that fixes remain effective as the platform scales. • You stay accountable for the outcome • Sezzle’s Technology Stack: • Languages: Golang, Python • Languages: • Database: MySQL, Postgres • Database: • DevOps and Cloud: AWS, Kubernetes • DevOps and Cloud: • Version Control: Git • Version Control: • CI/CD: • Open Source: Sezzle is focused on using open source, and we build what we can before buying! • Open Source: • What Makes Working at Sezzle Awesome? • At Sezzle, we are more than just brilliant engineers, passionate data enthusiasts, out-of-the-box thinkers, and determined innovators; we are skilled musicians, yogis, cyclists, chefs, golfers, dog-lovers, and rock-climbers. We believe in surrounding ourselves with not only the best and the brightest individuals, but those that are unique and purpose-driven in all that they do. Our culture is not defined by a certain set of perks designed to give the illusion of the traditional startup culture, but rather, it is the visible example living in every employee that we hire. • #Li-remote #full-time
Responsibilities
• Own the technical architecture and evolution of core infrastructure: identify system limits, prioritize technical improvements, and implement changes that support increasing traffic, data volume, and workload complexity. • Own the technical architecture and evolution of core infrastructure • Connect business understanding to improvements across the system: learn how key business workflows behave in production, trace their impact across applications, data, and infrastructure, and partner across teams to improve performance, reliability, and cost efficiency beyond any single service or team's scope. • Connect business understanding to improvements across the system • Engineer for scale and performance: build capacity models, run load and stress tests, diagnose bottlenecks across compute, networking, Kubernetes, and databases, and validate improvements against throughput, latency, saturation, and cost per workload. • Design and build AWS infrastructure: implement resilient account, IAM, network, and service architectures; address service quotas, fault isolation, and multi-AZ or multi-region requirements as workloads grow. • Design and build AWS infrastructure • Build and operate the Kubernetes platform: improve cluster architecture, lifecycle automation, workload isolation, resource allocation, autoscaling, safe upgrades, and deployment reliability. • Build and operate the Kubernetes platform • Scale and optimize Aurora RDS for MySQL and Postgres: tune queries and indexes, address connection and replication bottlenecks, plan capacity, improve failover behavior, and implement safe schema changes and database migrations with application engineers. • Scale and optimize Aurora RDS for MySQL and Postgres • Improve reliability through engineering: define and instrument service-level objectives and error budgets with service owners; implement failure isolation, backpressure, load shedding, and safe retry behavior where needed to prevent cascading failures. • Improve reliability through engineering • Participate in on-call and drive technical recovery during serious incidents: use evidence-based triage, execute mitigations, communicate findings, and implement corrective actions from postmortems. • Participate in on-call and drive technical recovery during serious incidents • Implement and test disaster recovery: design backup, restore, and failover mechanisms against agreed recovery time and recovery point objectives; run recovery exercises and document measured results. • Implement and test disaster recovery • Build infrastructure-as-code and operational automation: make provisioning, configuration, deployments, upgrades, and recovery reproducible, reviewed, testable, and recoverable. Eliminate recurring manual work through code. • Build infrastructure-as-code and operational automation • Build observability that makes production diagnosable: improve metrics, logs, traces, dashboards, and actionable alerts, with visibility into service health, scaling limits, and customer impact. • Build observability that makes production diagnosable • Deliver safe infrastructure migrations: design phased rollouts, compatibility checks, validation, and rollback paths for changes to shared production systems. • Deliver safe infrastructure migrations • Improve cloud cost efficiency through technical changes: right-size resources, improve utilization, tune autoscaling and storage, and quantify savings while maintaining reliability and performance targets. • Improve cloud cost efficiency through technical changes • Build and evaluate AI-assisted operational tooling: apply AI to investigation, runbooks, anomaly analysis, and repetitive operations, with bounded permissions, reviewable actions, and measurable improvements in accuracy or toil. • Build and evaluate AI-assisted operational tooling • Make technical decisions clear and executable: write architecture proposals, evaluate technology tradeoffs through prototypes and benchmarks, review changes affecting shared infrastructure, and document how systems operate and fail. • Make technical decisions clear and executable • What We look for: • What We look for: • Bachelor's degree in Computer Science or a similar technical field (required) • 12+ years of experience across infrastructure, platform, site reliability, software development, or related engineering disciplines, with substantial hands-on depth designing and operating production infrastructure at scale. • 12+ years • Deep expertise with AWS: production experience across compute, IAM, multi-account architectures, and networking, including VPC design and private connectivity. • Deep expertise with AWS • Deep expertise with Kubernetes in production: cluster lifecycle, scheduling, resource management, autoscaling, networking, and troubleshooting business-critical workloads. EKS experience is strongly preferred. • Deep expertise with Kubernetes in production • Deep expertise with relational databases at scale, specifically RDS/Aurora (MySQL and/or Postgres): query performance, indexing, connection management, replication, high availability, failover, and verified backup and recovery. • Deep expertise with relational databases at scale • RDS/Aurora (MySQL and/or Postgres) • A track record of personally delivering infrastructure scaling improvements: identifying constraints, measuring baseline behavior, implementing changes, and demonstrating gains in capacity, latency, reliability, or cost efficiency. • A track record of personally delivering infrastructure scaling improvements • Strong coding and automation skills, using Golang, Python, or similar languages to build production tooling and eliminate operational toil, alongside infrastructure-as-code experience with Terraform or equivalent.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT