Data Mesh Paradigm Shift in Data Platform Architecture
Arif Wider at DDD Europe 2020
Many enterprises are investing in their next generation data platform, with the hope of democratizing data at scale to provide business insights and ultimately make automated intelligent decisions. Data platforms based on the data lake architecture have common failure modes that lead to unfulfilled promises at scale.
In this talk Arif shares his observations on the failure modes of a centralized paradigm of a data lake, or its predecessor data warehouse.
He introduces Data Mesh, the next generation data platforms, that shifts to a paradigm that draws from modern distributed architecture: considering domains as the first class concern, applying platform thinking to create self-serve data infrastructure, and treating data as a product.
Dr. Arif Wider is a lead developer at ThoughtWorks, where he enjoys building scalable applications and building teams that build scalable applications. As the Head of AI for ThoughtWorks Germany he is leading the data and machine learning activities. He is still passionate about functional programming and software engineering excellence.
In this talk Arif shares his observations on the failure modes of a centralized paradigm of a data lake, or its predecessor data warehouse.
He introduces Data Mesh, the next generation data platforms, that shifts to a paradigm that draws from modern distributed architecture: considering domains as the first class concern, applying platform thinking to create self-serve data infrastructure, and treating data as a product.
Dr. Arif Wider is a lead developer at ThoughtWorks, where he enjoys building scalable applications and building teams that build scalable applications. As the Head of AI for ThoughtWorks Germany he is leading the data and machine learning activities. He is still passionate about functional programming and software engineering excellence.