Black Duck Software, Inc. - Senior Data Engineer
Upload My Resume
Drop here or click to browse · PDF, DOCX, DOC, RTF, TXT
Requirements
• Solid experience building and operating production data pipelines and datasets in a cloud environment. • Strong SQL skills and working Python experience for building data pipelines, transformations, and automation. • Handson experience with cloud platforms (AWS or Google Cloud), including core services such as compute, storage, IAM, and networking. • Experience working with both operational databases (e.g., RDSstyle systems) and analytics or columnar data stores.
Responsibilities
• Design and build data pipelines and datasets supporting one or more product areas within an established, governed data platform. • Implement ingestion patterns for batch, streaming, and event data, onboarding new data sources using existing frameworks and standards. • Build and maintain reliable data pipelines with appropriate data quality checks, basic lineage, monitoring, and alerting. • Develop curated datasets and views that are secure, discoverable, and ready for analytics, reporting, and downstream consumption. • Contribute to customerfacing data products such as datasets, metrics, exports, or feeds, ensuring accuracy and clear documentation. • Implement data models for both operational databases and analytics stores, following agreed modelling conventions and best practices. • Support ML and analytics use cases by producing trusted training datasets and featureready outputs with consistent definitions. • Optimize pipeline performance, cost, and reliability within defined architectural patterns. • Collaborate with product managers, engineers, analysts, and security partners to deliver data solutions aligned to roadmap needs. • Participate in oncall rotations and operational support for data services owned by the team. • Contribute to technical discussions, code reviews, and continuous improvement of data engineering practices.
Similar Jobs
No credit card. Takes 10 seconds.