G-P - Senior Software Engineer (Data Platform)
Requirements
• Scale Specialist: 5+ years of experience building and operating production-grade data systems at massive scale. • Scale Specialist: • Databricks Expert: Deep, hands-on mastery of the Databricks/Spark ecosystem (Delta Lake, DLT, Spark UI debugging, and performance tuning). • Databricks Expert: • Databricks/Spark • Streaming Veteran: Proven track record of building Real-time/Streaming architectures (Spark Structured Streaming, Kafka, or Kinesis) as a core production requirement. • Streaming Veteran: • Real-time/Streaming • Optimisation Mindset: Experience managing and optimizing cloud costs in a high-growth environment. • Optimisation Mindset: • Platform Thinker: Experience building APIs, tools, or frameworks used by other internal engineering teams. • Platform Thinker: • Actual compensation for this position may vary and will depend on multiple factors such as relevant qualifications, experience, education, and geographic location. For Full-Time Regular Employees, this position is also eligible for additional compensation as follows: • Sales Roles: This position is eligible for a commission structure in addition to base salary. • Non-Sales Roles: This position is eligible for an annual bonus which is paid dependent on various factors, including and without limitation, individual and company performance in addition to base salary.
Responsibilities
• Architect the Data Platform • Lead the design and implementation of internal SDKs and self-service frameworks that enable distributed engineering teams to ingest and transform data autonomously. • internal SDKs and self-service frameworks • Shift from "pipeline building" to "platform engineering," creating reusable patterns for batch and real-time event processing. • "platform engineering," • Own Platform Performance • Take full ownership of the cost-effectiveness of the Databricks ecosystem. You will tune Spark execution plans, optimize shuffle partitions, and implement auto-scaling strategies to manage DBU consumption. • cost-effectiveness • Ensure the platform remains performant as volume grows, managing the trade-offs between latency, throughput, and cloud spend. • Drive Data Contracts & Governance • Implement Schema-on-Write validation and Data Contracts to ensure data from hundreds of internal services meets strict quality standards before hitting the Bronze layer. • Schema-on-Write validation • Data Contracts • Partner with the Data Architect & Data Stewards to enforce data privacy (PII), security standards, and metadata lineage across the global ecosystem. • Lead an AI-Infused SDLC • Champion the use of AI-assisted development tools (e.g., GitHub Copilot, Cursor) to accelerate the engineering lifecycle and improve code quality. • AI-assisted development tools • Mentor engineers on distributed computing best practices, conducting deep-dive code reviews that focus on scalability and maintainability.
Benefits
• G-P values its employees and offers excellent benefits and perks including generous paid parental leave, flexible time off, spending accounts, medical insurance, dental insurance, vision insurance, sabbatical after 5 years and more. • Individuals residing, or applying to work, in the United States: California or Philadelphia, Pennsylvania, please review the following additional information:G-P will consider qualified applicants with arrest or conviction records in accordance with the California Fair Chance Act, Los Angeles City Fair Chance Act Ordinance, Los Angeles County Fair Chance Act Ordinance, and San Francisco Fair Chance Act Ordinance. Los Angeles applicants can review additional information regarding the Los Angeles City Fair Chance Act here: Fair Chance Initiative for Hiring Ordinance, and Philadelphia applicants can review information pertaining to Philadelphia’s Fair Criminal Record Screening Standards Ordinance here: Fair Chance Poster. Any consideration of a candidate’s background check with arrest or conviction records will include an individualized assessment based on the factors required by applicable law, including the candidate’s specific record and the duties and requirements of the specific job. • United States: California or Philadelphia • Pennsylvania, • G-P. Global Made Possible.
Apply in one click
Upload My Resume
Drop here or click to browse · Tap to choose · PDF, DOCX, DOC, RTF, TXT