Principal Data Engineer
About this opportunity
Career Category
Engineering
Job Description
Job Description
ABOUT THE ROLE
Role Description:
Let’s do this. Let’s change the world. We are looking for highly motivated expert Principal Data Engineer who can own the design & development of complex data pipelines, solutions and frameworks. The ideal candidate will be responsible to design, develop, and optimize data pipelines, data integration frameworks, and metadata-driven architectures that enable seamless data access and analytics. This role prefers deep expertise in big data processing, distributed computing, data modeling, and governance frameworks to support self-service analytics, AI-driven insights, and enterprise-wide data management.
Roles & Responsibilities:
Architect and maintain robust, scalable data pipelines using Databricks, Spark, and Delta Lake, enabling efficient batch and real-time processing.
Lead efforts to evaluate, adopt, and integrate emerging technologies and tools that enhance productivity, scalability, and data delivery capabilities.
Drive performance optimization efforts, including Spark tuning, resource utilization, job scheduling, and query improvements.
Identify and implement innovative solutions that streamline data ingestion, transformation, lineage tracking, and platform observability.
Build frameworks for metadata-driven data engineering, enabling reusability and consistency across pipelines.
Foster a culture of technical excellence, experimentation, and continuous improvement within the data engineering team.
Collaborate with platform, architecture, analytics, and governance teams to align platform enhancements with enterprise data strategy.
Define and uphold SLOs, monitoring standards, and data quality KPIs for production pipelines and infrastructure.
Partner with cross-functional teams to translate business needs into scalable, governed data products.
Mentor engineers across the team, promoting knowledge sharing and adoption of modern engineering patterns and tools.
Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals.
Stay up to date with emerging data technologies and best practices, ensuring continuous improvement of Enterprise Data Fabric architectures.
Must-Have Skills:
Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies.
Proficiency in workflow orchestration, performance tuning on big data processing.
Strong understanding of AWS services
Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures.
Ability to quickly learn, adapt and apply new technologies
Strong problem-solving and analytical skills
Excellent communication and teamwork skills
Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices.
Good-to-Have Skills:
Good to have deep expertise in Biotech & Pharma industries
Experience in writing APIs to make the data available to the consumers
Experienced with SQL/NOSQL database, vector database for large language models
Experienced with data modeling and performance tuning for both OLAP and OLTP databases
Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops
Education and Professional Certifications
12 to 17 years of experience in Computer Science, IT or related field
AWS Certified Data Engineer preferred
Databricks Certificate preferred
Scaled Agile SAFe certification preferred
Soft Skills:
Excellent analytical and troubleshooting skills.
Strong verbal and written communication skills
Ability to work effectively with global, virtual teams
High degree of initiative and self-motivation.
Ability to manage multiple priorities successfully.
Team-oriented, with a focus on achieving team goals.
Ability to learn quickly, be organized and detail oriented.
Strong presentation and public speaking skills.
.
Job details
How this role compares
Computed from every other active Data & Digital role in our database, not just this employer's listings.
We currently track 93 comparable Principal Data & Digital roles across 25 biopharma companies.
Salary context
23 of 93 peers report a salary range (USD, annualized)
Peers share this role's job function and a matching or adjacent seniority level -- not necessarily the same therapeutic area or country.
Where these roles are based
Top locations among the 93 comparable roles
+ 5 more countries
Seniority mix
93 of 93 peers have a known seniority level
Therapeutic area mix
2 of 93 peers have a known therapeutic area; the rest are genuinely unlabeled, not hidden
Similar opportunities
The closest matches from our peer group, ranked by how similar they are, not how well you'd qualify for them -- treat this as market context, not a guaranteed shortlist; a weak match is labeled as one below.
Notify me about similar jobs
Get an email when we spot other openings like this one – same job function, comparable seniority, roles you'd actually want to see.
How we calculate "similar"
No black box, no LLM guesswork: a deterministic score built from four normalized attributes. Here's this role's own peer group at different match levels, so you can see the mechanism, not just the result.
Every comparison starts from the same 100-point budget: 25 for working in the same function, 40 for the same therapeutic area, 20 for the same or adjacent seniority, 15 for the same country. A dimension we can't confirm on both sides contributes nothing, never a guess, never a free pass.
0 points, never a partial guess. A role we know almost nothing about beyond its function bottoms out at 25%; it never inflates to 100% just because there's little to compare against. Seniority uses a defined ladder (Associate → Manager → Associate Director → Senior → Principal → Director → Senior Director → Executive/VP) so "Director" and "Senior Director" count as adjacent, but "Director" and "Executive/VP" do not.