Senior Data Engineer
Upload My Resume
Drop here or click to browse · PDF, DOCX, DOC, RTF, TXT
Requirements
• 6+ years working in a data engineering role supporting incoming/outgoing products for internal and external customers. • 4+ years demonstrated expertise in designing an end-to-end data pipeline in cloud frameworks (such as GCP, AWS, Azure) with requirements from multiple stakeholders. • 4+ years of Python experience writing efficient, testable, and readable code • 2+ years of experience in building streaming data ingestion pipelines • 1+ year of ML (Machine Learning) support and implementation or MLOps. • Advanced SQL expertise with columnar databases (BigQuery, Snowflake, Amazon Redshift) and performance tuning. • Demonstrated experience with AI Coding assistants – AI tools are heavily engrained in Bestow culture. • Cloud Native: Deep experience with cloud services (GCP preferred: Cloud Run, Pub/Sub, BigQuery) and containerization (Docker/Kubernetes). • Orchestration: Expert-level knowledge of Apache Airflow (DAG optimization, custom operators). • Experience building CICD pipelines for data processing using tools such as Docker, CircleCI, dbt, git, etc • Infrastructure as Code: Proven experience managing infrastructure using Terraform or Pulumi. • Experience with creating alerts and monitoring pipelines which contribute to overall data governance. • Familiarity with standard IT security practices such as identity and access management (IAM), data protection, encryption, certificate, and key management. • Adaptability to learn new technologies and products as the job demands. • Nice to have: Familiarity with building tools that draw upon Generative AI (GenAI) integrations (Enterprise-grade, not simply vibe-coded). • Nice to have: experience with data contracts, data lakes, and API development
Responsibilities
• Transform and build robust solutions for transferring data from first and third-party applications to and from our data warehouse • Envision and design toward industry patterns on data exchange with our enterprise clients through a mix of traditional push delivery, cloud, and event-driven (eg: API, grpc) data sharing methods. • Support implementation and data integration as new partners roll onto the Bestow platform, and recommend improvements on configurability of platform. • Making decisions as a team. The things you build will be maintained and improved upon by others; there is a shared responsibility to make defensible design considerations and high collaboration. • Champion test-first design principles, proactively writing tests before code to maintain high coverage and pipeline reliability. • Develop hardened and repeatable (CI/CD) data models and pipelines to enable reporting, modeling and machine learning. • Ensure data quality through automated monitoring and alerting, and occasionally serving within an on-call rotation. • Leverage Google Cloud (GCP) tools (eg: Cloud Run, Cloud Functions, Vertex AI, App Engine, Cloud Storage, IAM, etc.) and services (eg: Astronomer - Apache Airflow) to bring data workloads to production. • Drive and support MLOps to improve Data Science monitoring and governance • Enable and support Generative AI (eg: LLM) pipelines, allowing internal teams to quickly prototype. Support the architecture and rollout of GenAI products and features into the marketplace. • Collaborate with product, engineering, stakeholders and data teams to deliver informed solutions to platform and client needs
Benefits
• $134.5K – $158.5K • Offers Equity • Upload your resume here to autofill key application fields. • Drop your resume here! • Parsing your resume. Autofilling key fields... • or drag and drop here • Note: we ask for address as we might need it for documents such as offer letters. This information, along with other contact information is kept private. • Including portfolio links and relevant passwords if applicable • No, I will not need immigration support now or in the future (U.S. citizen, Green Card holder, or other permanent resident status). • Yes, I will need a transfer of my current visa (such as an H-1B). • Yes, I will need a different type of sponsorship in the future (for example, OPT to H-1B). • Yes, I will need a TN visa. • Decline to self-identify • Hispanic or Latino - A person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture or origin regardless of race. • Hispanic or Latino • White (Not Hispanic or Latino) - A person having origins in any of the original peoples of Europe, the Middle East, or North Africa. • White • Black or African American (Not Hispanic or Latino) - A person having origins in any of the black racial groups of Africa. • Black or African American • Native Hawaiian or Other Pacific Islander (Not Hispanic or Latino) - A person having origins in any of the peoples of Hawaii, Guam, Samoa, or other Pacific Islands. • Native Hawaiian or Other Pacific Islander • Asian (Not Hispanic or Latino) - A person having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian Subcontinent, including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam. • Asian • American Indian or Alaska Native (Not Hispanic or Latino) - A person having origins in any of the original peoples of North and South America (including Central America), and who maintain tribal affiliation or community attachment. • American Indian or Alaska Native • Two or More Races (Not Hispanic or Latino) - All persons who identify with more than one of the above five races. • Two or More Races • Hispanic or Latino • White (Not Hispanic or Latino) • Black or African American (Not Hispanic or Latino) • Native Hawaiian or Other Pacific Islander (Not Hispanic or Latino) • Asian (Not Hispanic or Latino) • American Indian or Alaska Native (Not Hispanic or Latino) • Two or More Races (Not Hispanic or Latino) • I identify as one or more of the classifications of protected veteran listed above • I am not a protected veteran