Senior Data Engineer
The Senior Data Engineer at Commvault will be responsible for designing, building, automating, and operating shared data and AI platform capabilities utilized by engineering, analytics, governance, and AI teams. This senior hands-on role involves establishing reusable platform patterns across Databricks, Microsoft Azure, Microsoft Fabric, CI/CD, observability, security, and cost management. The position also provides technical leadership for platform reliability, operational maturity, and developer enablement.
Key responsibilities include designing and implementing secure, scalable platform services for Databricks, Azure Data Lake, Microsoft Fabric, and supporting cloud services. The role involves creating reusable environment patterns for development, test, and production workloads, implementing workspace, network, identity, storage, compute, catalog, policy, and secrets-management configurations. Collaboration with Architecture and Security teams to translate enterprise standards into practical platform controls and reference implementations is essential.
The Senior Data Engineer will develop infrastructure-as-code modules, deployment pipelines, automated configuration, and policy-as-code controls. Establishing CI/CD patterns for data pipelines, notebooks, libraries, semantic assets, and AI workloads is a key aspect of the role. Building self-service templates and paved-road capabilities to improve delivery consistency and reduce manual provisioning is also expected. Maintaining source-control, release, artifact, environment-promotion, and rollback practices is crucial.
The role requires defining and implementing monitoring, alerting, logging, capacity, performance, availability, and recovery standards. Leading troubleshooting and root-cause analysis for complex platform incidents and recurring reliability issues is part of the responsibilities. Implementing operational dashboards, service-level indicators, runbooks, and on-call support patterns is expected. Analyzing platform usage and cost drivers, implementing guardrails, tagging, budgets, and optimization recommendations are also key tasks.
Technical leadership and governance are integral to the role, including leading platform design reviews and providing technical guidance to Platform Engineers and delivery teams. Developing engineering standards, operating procedures, support models, and platform documentation is essential. Embedding data governance, lineage, classification, access control, and audit requirements into platform capabilities is expected. Evaluating platform features and emerging technologies, recommending pragmatic adoption paths aligned with business outcomes, is also part of the role.