Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

ME05c1R5eVA5RlJ2ZUYxb0dUN2t4dnRzenc9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

Hampton Inn Viera

Hotel Housekeeper Job at Hampton Inn Viera

Join Our Team as a Housekeeper at Hampton Inn Viera! Where Clean Spaces Meet Friendly Faces Wage & Benefits Youll Love: We believe in rewarding hard work with great perks: ~ Competitive Pay ~ Medical, Dental & Vision Insurance ~401(K) with Company Match ...

McGrath Honda Elgin

Product Specialist / Sales Professional Job at McGrath Honda Elgin

 ...plenty of room for growth, and we are here to help you!If you are looking for a CAREER PATH, we want to speak to you! No experience necessary, we will train you! **Entry Level Product Specialist** (no experience needed- we will train you!) - 40 hours a week | $2,5... 

Safety Management Systems

Offshore Drilling HSE Specialist Job at Safety Management Systems

 ...Safety Management Systems is seeking Offshore Drilling HSE Specialists to provide HSE support for projects in the Gulf of Mexico. The HSE Consultant will work with the company and contractor's site management team to develop and implement the project and site safety... 

UrgentVet

Urgent Care Veterinary Practice Manager Job at UrgentVet

 ...Department: UrgentVet Hospitals Location: UrgentVet - Sarasota Description Are you an experienced Veterinary Practice Manager with a passion for team leadership and veterinary business operations? UrgentVet is seeking a dynamic Practice Manager to lead our... 

Future Publishing

Senior Planner, Media Planning & Strategy Job at Future Publishing

 ...Description Are you a media planning expert within the digital media space who is also passionate about the latest innovations in advertising? Are you motivated by the challenge of building optimized and strategic media plans based on client goals? If so, we want to...