Sagar Mangam
Practice ArchitectAvaap
Biography:
Sagar has spent nearly two decades working with big data in complex systems. He is currently a practice data engineer, solution architect and thought leader for Avaap - a management and technology consulting firm that specializes in data and analytics. He has extensive experience building data pipelines, architecting data platforms and coordinating data & analytics projects. Sagar can be found doing some light reading on PySpark fundamentals or scoring a century on the cricket pitch.
Session(s)
Data engineers must be adept in several areas. With years of SME experience on conventional tools, moving data to Hadoop will be a steep learning curve. Hadoop platform also offer a wide variety of services that are custom built to serve a specific purpose. A data engineer is expected to integrate with assortment of data sources at scale, read & stage data, transform and load data across on-premises and cloud ecosystems amongst the specified purpose built targets. StreamSets provides us with the capability to reduce the barrier for developers to adopt to a distributed ecosystem, load and transform data on big data ecosystems. Re-usable & parameterized pipelines, fragments, connections and service specific modules improve the usability of the pipelines.