Brillio
182 open roles · AI & Data Engineering
Databricks Data Specialist - R01569707
Apply on company siteYou’ll apply directly with the employer on Lever.
- Added to ZestAmigo
- Last seen on employer site
Where you can work
Hybrid
Bengaluru, Karnataka, India
View location wording from the posting
Bangalore, Karnataka, India
Role overview
Extracted from the posting. Read the employer’s full description below.
- Delta Lake
- Delta Live Tables (DLT)
- Unity Catalog
- Databricks Workflows
- PySpark and Apache Spark
- Structured Streaming
- Auto Loader
- SQL
- Lakehouse Data Modeling
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Microsoft Purview
- Microsoft Fabric
- AWS Glue
- AWS Lambda
- AWS Step Functions
- Apache Airflow
- DBT
- Fivetran
- Informatica
- Apache Kafka
- Power BI
- Collibra
- Alation
- BigQuery
- Experience working in agile and collaborative environments.
- Excellent communication and stakeholder management skills.
What they are looking for
- Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark .
- Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
- Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
- Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
- Implement scalable and efficient data ingestion processes using Auto Loader .
Show all 46 itemsShow fewer items
- Develop and manage real-time data processing solutions using Structured Streaming .
- Orchestrate, schedule, and monitor data workflows using Databricks Workflows .
- Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
- Establish and enforce data governance, security, and access controls using Unity Catalog .
- Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
- Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
- Delta Lake
- Delta Live Tables (DLT)
- Unity Catalog
- Databricks Workflows
- PySpark and Apache Spark
- Structured Streaming
- Auto Loader
- SQL
- Lakehouse Data Modeling
- Strong understanding of data engineering best practices and scalable data architectures
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Microsoft Purview
- Microsoft Fabric
- AWS Glue
- AWS Lambda
- AWS Step Functions
- Apache Airflow
- DBT
- Fivetran
- Informatica
- Apache Kafka
- Power BI
- Collibra
- Alation
- BigQuery
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
- Strong analytical, troubleshooting, and problem-solving capabilities.
- Experience working in agile and collaborative environments.
- Excellent communication and stakeholder management skills.
- Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
- Strong understanding of data governance, security, and compliance frameworks.
- Experience delivering both batch and real-time data processing solutions.
- Ability to work independently while collaborating effectively across global teams.
Employer description
Data Specialist
Primary Skills
Databricks Engineer
Role Overview
We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
- Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
- Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
- Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
- Implement scalable and efficient data ingestion processes using Auto Loader.
- Develop and manage real-time data processing solutions using Structured Streaming.
- Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
- Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
- Establish and enforce data governance, security, and access controls using Unity Catalog.
- Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
- Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
Required Skills (Must Have)
- Databricks Platform
- Delta Lake
- Delta Live Tables (DLT)
- Unity Catalog
- Databricks Workflows
- PySpark and Apache Spark
- Structured Streaming
- Auto Loader
- SQL
- Lakehouse Data Modeling
- Strong understanding of data engineering best practices and scalable data architectures
Preferred Skills (Good to Have)
Azure Ecosystem
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Microsoft Purview
- Microsoft Fabric
AWS Ecosystem
- AWS Glue
- AWS Lambda
- AWS Step Functions
Data Engineering & Integration
- Apache Airflow
- DBT
- Fivetran
- Informatica
Streaming & Analytics
- Apache Kafka
- Power BI
Data Governance
- Collibra
- Alation
GCP
- BigQuery
Qualifications
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
- Strong analytical, troubleshooting, and problem-solving capabilities.
- Experience working in agile and collaborative environments.
- Excellent communication and stakeholder management skills.
Preferred Candidate Profile
- Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
- Strong understanding of data governance, security, and compliance frameworks.
- Experience delivering both batch and real-time data processing solutions.
- Ability to work independently while collaborating effectively across global teams.
Key Technologies
Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
Specialization
- Databricks Engineering: Lead Data Engineer
Job requirements
Databricks Engineer
Role Overview
We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
- Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
- Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
- Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
- Implement scalable and efficient data ingestion processes using Auto Loader.
- Develop and manage real-time data processing solutions using Structured Streaming.
- Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
- Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
- Establish and enforce data governance, security, and access controls using Unity Catalog.
- Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
- Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
Required Skills (Must Have)
- Databricks Platform
- Delta Lake
- Delta Live Tables (DLT)
- Unity Catalog
- Databricks Workflows
- PySpark and Apache Spark
- Structured Streaming
- Auto Loader
- SQL
- Lakehouse Data Modeling
- Strong understanding of data engineering best practices and scalable data architectures
Preferred Skills (Good to Have)
Azure Ecosystem
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Microsoft Purview
- Microsoft Fabric
AWS Ecosystem
- AWS Glue
- AWS Lambda
- AWS Step Functions
Data Engineering & Integration
- Apache Airflow
- DBT
- Fivetran
- Informatica
Streaming & Analytics
- Apache Kafka
- Power BI
Data Governance
- Collibra
- Alation
GCP
- BigQuery
Qualifications
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
- Strong analytical, troubleshooting, and problem-solving capabilities.
- Experience working in agile and collaborative environments.
- Excellent communication and stakeholder management skills.
Preferred Candidate Profile
- Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
- Strong understanding of data governance, security, and compliance frameworks.
- Experience delivering both batch and real-time data processing solutions.
- Ability to work independently while collaborating effectively across global teams.
Key Technologies
Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
Track this application
Keep your own notes. Only you can mark an application as sent.
Report a problem with this listing
Sign in to report this listing.
More at Brillio
Hybrid
Andhra Pradesh, India
Lead
Salary unavailable
Posted 1 week ago
Source: Lever
Hybrid
Andhra Pradesh, India
Lead
Salary unavailable
Posted 1 week ago
Source: Lever
Hybrid
Gurugram, Haryana, India
Lead
Salary unavailable
Posted 1 week ago
Source: Lever
Hybrid
Andhra Pradesh, India
Senior
Salary unavailable
Posted 5 days ago
Source: Lever