Change Data Capture stage is a stage which takes sorted input on the key fields and generates a change code which denotes the type of change. Based on change code DMLs can be performed on target. CDC needs sorted input hence if you have large number of records to compare against then it could impact the performance of the job. Also this kind of job could be very memory intensive hence its placement in the sequence should be correct during the execution. While using CDC DO NOT forget to drop the change_code field from the schema of the datasets otherwise the next job to load data in the database would generate error.
Change Data Capture stage is a stage which takes sorted input on the key fields and generates a change code which denotes the type of change. Based on change code DMLs can be performed on target. CDC needs sorted input hence if you have large number of records to compare against then it could impact the performance of the job. Also this kind of job could be very memory intensive hence its placement in the sequence should be correct during the execution. While using CDC DO NOT forget to drop the change_code field from the schema of the datasets otherwise the next job to load data in the database would generate error.
ReplyDelete