Want your enterprise to scale out while containing cost? Customer Data Integration is a core Master-Data-Management (MDM) function for most enterprises: being intentional about where to perform your most resource-consuming data management processes, and choosing the right open-source columnar data store, can help. In this talk we will look at the migration of Allant's CDI-keying engine, built using JMS and Oracle, to an architecture centered around Cassandra as the data store.
Attendees will learn how a legacy Oracle-centric pipeline can be migrated to a Cassandra-based solution by:
1) using Spark to push the bulk of the functions originally executed at the application layer down to the distributed data layer and
2) using Cassandra and Zookeeper to migrate in-process locking mechanisms to a scalable system; and
3) using Spark to create an high-throughput ETL pipeline for loading large datasets into Cassandra.
Allant's original CDI engine was architected and designed to accommodate both batch and real time inputs, with a common API to ensure unique creation of customer information. The common API is facilitated via a batch-to-single interface (proprietary, custom software) that drops input records into JMS to perform the parsing, hygiene, matching, and keying sub-functions. This pipeline was re-implemented by using Cassandra as a scalable, high-throughput data store, and Spark to achieve distributed function execution while maintaining the custom fuzzy matching logic of the original implementation.
Fausto Inestroza - Silicon Valley Data Science
Combining his expertise of emerging technologies and cross-industry experience, Fausto helps clients architect big data platforms and build data-driven products. He has extensive experience with data platforms, analytical processes, and distributed systems.
Shambho Krishnasamy - The Allant Group
Shambho joined Allant in May 2014. He leads the architecture team and is responsible for Allant's overall Product & Services technology stack, architecture and product design. He also works closely with clients in product and solution delivery. He has 20+ years of experience in product development, consulting and CRM industries with specific domain expertise in architecting and delivering large scale database marketing and digital CRM solutions for clients across Retail, Insurance, Life-Sciences, Financial and Media & Entertainment verticals. Prior to Allant, Shambho was VP, Chief Solution Architect at Merkle and prior to that he was VP, Principal Architect at Harte-Hanks. He has an undergraduate degree in Applied Sciences and Masters Degree in Computer Applications.
