Hi,
Are you facing extremely slow performance in Oracle Connector when you try to update a table in Oracle using column that has (NUMBER) data type as the Filtering condition (where clause)?
My friend, look no further this is a common problem me and my team faced using the latest Datastage version. This Datastage double data type even locks the table in addition to being extremely slow.
Why is this happening? In Oracle the NUMBER data type without any precision or scale is kind of an all purpose float datatype that can store any sorts of number from negative digits to numbers with lots decimal places.
This causes Datastage to interpret it as DOUBLE data type is pretty much a floating numeric datatype.
During Inserts you will not notice any performance hit
However during updates or Deletes where the filtering condition uses a column with double datatypes. You will notice how slow and nonfunctional it is..
To fix the situation we decided to remap the column definition of the Oracle table from Datastage Double datatype to Decimal (38,10). This way we are sure not to run into this issue. Converting it from a floating datatype to Decimal which is more precise boosts the performance and prevents locks on the table.
This can be easily done in the Table definition import by using Orchestrate plug in for Oracle instead of the wizard
Subscribe to:
Post Comments (Atom)
BECOME A BIG DATA ENGINEER IN 3 MONTHS with less than $100 investment
INTRO: Below is my guide to becoming a data engineer based on the current job market (08/08/2020) demands. I have outlined the TOP 5 foun...
-
1. Good Foundational Knowledge of SQL Programming (SQL Query writing) Paid Course The best SQL course that I know. (I have not affili...
-
Hi I was recently working on extracting a table that has 400 columns from a SQL Server DB to load into an Oracle Target Database. I design...
-
In this blog post I will show you how to implement your own custom logging in Azure Data Factory Before you do this please note that Azur...
No comments:
Post a Comment