Agile Data Warehousing: Building a Virtualized ODS

Presented by: Kent Graziano, Senior Technical Evangelist at Snowflake Computing

At a past client, in order to meet timelines to fulfill urgent, unmet reporting needs, we found it necessary to build a virtualized Operational Data Store (ODS) as the first phase of a new Data Vault 2.0 project. This allowed us to deliver new objects, quickly and incrementally to the report developer so we could quickly show the business users their data. In order to limit the need for refactoring in later stages of the data warehouse development, we chose to build this virtualization layer on top of a Type 2 persistent staging layer. All of this was done using Oracle SQL Developer Data Modeler (SDDM) against (gasp!)  a MS SQL Server Database. In this talk I will show you the architecture for this approach, the rationale, and then some of the tricks I used in SDDM to build all the stage tables and views very quickly. In the end you will see actual SQL code for the virtual ODS that you can leverage for your own projects.

Goals:

• learn a data warehouse architecture that can be implemented incrementally and in layers
• learn what a Virtual ODS is and how it can help you be Agile
• how to quickly build pattern based stage tables in SDDM
• how to build complex, pattern based views in SDDM
• tip on how to use analytic functions to create virtual columns
• see examples of how to embed change data capture into a view-based ELT loading architecture

Speaker

Kent Graziano is a Senior Technical Evangelist with Snowflake Computing and the author of The Data Warrior blog. He is a certified Data Vault Master (CDVP2), Oracle ACE Director, former member of the Boulder BI Brain Trust (#BBBT), expert data modeler and architect with over 30 years of experience, including 25 years doing data warehousing with multiple architectures. Kent is an award winning speaker and author having written numerous articles, and has given hundreds of presentations, nationally and internationally. He is a co-author of four books including The Data Model Resource Book (1st Edition), and Supercharge Your Data Warehouse. His now has several ebooks on Kindle including A Check List for Doing a Data Model Design Reviews and An Introduction to Agile Data Engineering. You can follow Kent on his blog and twitter @kentgraziano.