Kosmos world

Thanks to my colleague Sobhan Surapaneni for helping me out with some of the details of CDC. Sobhan was the guy who made it all happen on the project and really knows this stuff cold. Introduction Traditional Data Warehousing and BI systems rely on batch data extraction and load routines (ETL or ELT) to acquire their data from sources. Depending on data timeliness needs, this may work for intervals down to perhaps one hour for small data sets. However as data timeliness needs go below what is reasonable with a batch process, a complete shift in technology is required. There are tools out there that acquire data in a “real-time” manner directly from the database log files and send individual transactions out over the wire to a target database. This post is about Informatica’s CDC product, but the lessons and the manner in which it works are similar for another popular product called Golden Gate from Oracle. Note the name Change Data Capture is not the best; this really is more abou...

View PDF <p>The script content on this page is for navigation purposes only and does not alter the content in any way.</p> Change Data Capture efficiently identifies and captures data that has been added to, updated in, or removed from, Oracle relational tables and makes this change data available for use by applications or individuals. Change Data Capture is provided as a database component beginning with Oracle9 i . This chapter describes Change Data Capture in the following sections: Overview of Change Data Capture Change Sources and Modes of Data Capture Change Sets Change Tables Getting Information About the Change Data Capture Environment Preparing to Publish Change Data Publishing Change Data Subscribing to Change Data Considerations for Asynchronous Change Data Capture Managing Published Data Implementation and System Configuration See PL/SQL Packages and Types Reference for reference information about the Change ...