Enterprise Data Consultant - Senior Collibra Lineage Developer

About the position

This is a remote role that may be hired in several markets across the United States. The Enterprise Data Consultant (Senior Collibra Lineage Developer) is responsible for designing and implementing data lineage solutions to ensure the traceability of data throughout its lifecycle. This role involves collaborating with data architects, data analysts, and other stakeholders to develop an understanding of data flows, transformations, and dependencies across various data systems.

Responsibilities

• You will be a member of a team whose purpose is to implement automated Lineage scanners and present the result via interactive Lineage diagrams.
• Your role will specifically aim to obtain lineage from BI & ETL tools.
• Your work will have a massive impact as we replace manual Lineage by an automated reverse engineering process.
• Analyzing data sources and working with Product team to define lineage scanner scope.
• Implementing algorithms and complex logic to extract lineage from sources like API outputs, XML files and others.
• Optimizing performance and participating in architectural design.
• Troubleshooting complex lineage processing pipelines.
• Cooperation with internal and external stakeholders and teams to create a well-integrated and customer-friendly product.
• Responsibilities include coding in Python, optimizing performance, and troubleshooting lineage processing workflows.

Requirements

• 5+ years of experience developing in Java and Spring Framework.
• Strong experience with relational databases.
• Experience in microservices architecture and hands-on with continuous integration and continuous deployment (CI/CD) pipelines.
• Strong familiarity with agile development methodologies and a proven command of code versioning tools, including Git.
• Familiarity with search technologies, particularly Elasticsearch is a bonus.
• A bachelor's degree or equivalent related working experience is required.
• Experience in Python and great motivation to have it as your primary programming language.
• Proficiency in data modeling tools and ETL platforms (E.g., DataStage, Informatica, SSIS, DBT etc.).
• Knowledge of the leading PaaS providers, Data Lakes, Databases, and Integration platforms (SQL, Databricks, GCP, Snowflake etc.).
• Experience with data visualization tools (e.g., Tableau, Power BI) to illustrate data lineage.
• Strong knowledge of SQL and database management systems (e.g., MySQL, PostgreSQL, Oracle).
• An agile mindset and like to work in scrum/kanban environments.
• Excellent analytical and problem-solving skills to identify data discrepancies.
• Strong communication skills to collaborate with cross-functional teams and stakeholders.
• Ability to work independently and manage multiple projects simultaneously.
• Leadership qualities to mentor junior team members and drive data lineage initiatives.
• Understanding of data governance and compliance regulations (e.g., GDPR, HIPAA).

Nice-to-haves

• Familiarity with search technologies, particularly Elasticsearch.

Benefits

• Competitive, thoughtfully designed and quality benefits program.