Industry leading client is seeking a Databricks Consultant for a contract opportunity in Fort Worth, TX!
Installation and Configuration
- Install/Configure Databricks in Azure
- Define Databricks, Spark, Java, Scala, PySpark versions
- Assess and recommend memory and CPU requirements
- Model the workload on the cluster
- Define and assess high data read and write volumes/storage
- Estimate the expected size.
- Define plan/procedures to use java/scala/pyspark
- Define security standards
- Define different user profiles and RBAC privileges using Azure AD
- Provide CI/CD pipelines
- EventHub, queue, log analytics, monitoring and other azure options for integrations
- Provide storage considerations.
Data modeling and application solution architecture
- Review and analyze existing data sources
- Review and provide assessment and recommendations for data model and application integration to support American data requirements and to support additional features
- Provide best practices for data modeling
- Provide architecture guidance and assessment.
Data ingestion assessment and optimization
- Assess and support updates to the current ingestion processing, while providing recommended best practices to include additional data sources and recommended best practices for recurrent updating of the existing data with updated data sources
- Review and provide recommendations and best practices to assure reliability and performance expectations are met
Visualization (define output)
- End user tooling considerations
- Query and code development and unit testing
- Performance testing and validation