NFQ - Senior Data Engineer

• Analyze and optimize business processes by collaborating with stakeholders to uncover inefficiencies and define data requirements for automation • Design scalable, modular data architectures that integrate with Generative AI and Agentic AI systems to support real-time decision-making • Engineer robust ETL/ELT pipelines using Python, cloud-native services, and orchestration tools, supporting both batch and streaming data needs • Architect RAG and vector database solutions using semantic search to enable LLMs to retrieve curated, context-rich business data • Build intelligent data products, from predictive models and decision engines to AI-driven insights platforms • Implement data quality, validation, and governance frameworks to ensure data integrity, lineage, and compliance across systems • Lead technical discovery sessions with clients to transform complex business challenges into AI and data-driven opportunities • Mentor team members on best practices in data engineering, AI integration, and modern cloud architectures • Expert-level Python proficiency for data engineering, including API integrations, data transformations (Pandas, PySpark), and automation • Proven experience designing and deploying large-scale data platforms on AWS, GCP, or Azure • Strong foundation in building production-grade ETL/ELT pipelines using Apache Airflow, Kafka, Spark, or cloud-native tools • Hands-on experience with vector databases (e.g., Pinecone, Weaviate, Chroma, Milvus) and implementing semantic search • Demonstrated knowledge of Generative AI and LLMs, with practical experience in RAG architectures and prompt engineering • Deep understanding of data governance, quality, and documentation, with a focus on lineage, metadata, and compliance • Familiarity with cloud services including serverless computing, managed databases, and data warehouses such as BigQuery, Redshift, or Snowflake • Experience working with complex real-world data environments, including legacy systems, SaaS integrations, APIs, and databases • Fluency in English, both written and spoken