Teradata mutliload CSV source file load issue

772 Views Asked by At

I have a comma separated CSV file that needs to be loaded into Teradata Table. Usually fastload is very useful in such scenarios when there is a single file which needs to be loaded into a single table. The issue here is that the data fields in CSV file are enclosed in quotes (Sample csv file data is shared below for reference). Fastload does not handle/support/load data when source file is in this format. TPUMP handles this type of data file but it creates macros in the background to issue statements. Unluckily the client for whom I am working does not provide 'Create Macro' privilege and thus TPUMP script is not an option. Can this file be loaded with the help of Multiload. If yes then would someone be kind enough to guide me on how to load this quoted data in teradata using mload. A small script example would be highly appreciated.

Thanks

"Received","From","Destination","Message Text","Status","Folder","Folder Owner",
"2020-06-25 13:31:47","125674122450","61119","EEEEEE","Not Processed","n/a","n/a"
"2020-06-25 13:14:15","125683011742","82332","CCC 000000 59303760","Processed","COMMERCIAL CCC","BSSSUser"
"2020-06-25 13:09:48","125693033666","61112","YesLLLL ","Processed","Cooooo AAAAA ","9890374"

@dnoeth I have written the below TPTLOAD Script but it is giving error



/**************************************************************/
/* IMI_T7_SMS_RESPONSES                                       */
/* Load CSV Data Into TERADATA                                */
/**************************************************************/
DEFINE JOB LOAD_TD_FROM_CSV
DESCRIPTION 'Load Teradata table from CSV File'
(
  DEFINE SCHEMA IMI_T7_SMS_RESPONSES
  DESCRIPTION 'IMI_T7_SMS_RESPONSES'
  (
    Response_Dttm             VARCHAR(100)
    ,From_Number              VARCHAR(20)
    ,To_Shortcode             VARCHAR(5)
    ,Response_Text          VARCHAR(100)
    ,Status                     VARCHAR(100)
    ,Folder                     VARCHAR(100)
    ,Folder_Owner           VARCHAR(100)
  );


  DEFINE OPERATOR DDL_OPERATOR
  TYPE DDL
  ATTRIBUTES
  (
   VARCHAR TdpId           = 'system',
   VARCHAR UserName        = 'username',
   VARCHAR UserPassword    = 'password'
   );


  DEFINE OPERATOR LOAD_CSV
  DESCRIPTION 'Operator to Load CSV Data'
  TYPE LOAD
  SCHEMA IMI_T7_SMS_RESPONSES
  ATTRIBUTES
  (
    VARCHAR PrivateLogName,
    VARCHAR TraceLevel      = 'None',
    INTEGER TenacityHours   = 1,
    INTEGER TenacitySleep   = 1,
    INTEGER MaxSessions     = 4,
    INTEGER MinSessions     = 1,
    VARCHAR TargetTable     = 'trm_lead_history.IMI_T7_SMS_RESPONSES_TEST',
    VARCHAR ErrorTable1     = 'trm_lead_history.IMI_T7_SMS_RESPONSES_E1',
    VARCHAR ErrorTable2     = 'trm_lead_history.IMI_T7_SMS_RESPONSES_E2',
    VARCHAR LogTable        = 'trm_lead_history.IMI_T7_SMS_RESPONSES_LOG',
    VARCHAR TdpId           = 'system',
    VARCHAR UserName        = 'User',
    VARCHAR UserPassword    = 'password'
  );

   DEFINE OPERATOR READ_CSV
   DESCRIPTION 'Operator to Read CSV File'
   TYPE DATACONNECTOR PRODUCER
   SCHEMA IMI_T7_SMS_RESPONSES
   ATTRIBUTES
  (
   VARCHAR  Filename             =   'C:\Users\nofel\Desktop\fastload\nick\IMI_T7_SMS_RESPONSES.csv'
   ,VARCHAR Format               =   'Delimited'
     ,VARCHAR TextDelimiter        =   ','
   ,VARCHAR CloseQuoteMark       =   '"'
   ,VARCHAR OpenQuoteMark        =   '"'
     ,VARCHAR AcceptExcessColumns  =   'N'
     ,VARCHAR PrivateLogName       =   'LOAD_FROM_CSV'
   ,VARCHAR QuotedData            =  'Y'
   ,Varchar EscapeQuoteDelimiter = '"'
  );


Step Setup_Tables
(
   APPLY
     ('DROP   TABLE TRM_LEAD_HISTORY.IMI_T7_SMS_RESPONSES_E1;'  ),
     ('DROP   TABLE TRM_LEAD_HISTORY.IMI_T7_SMS_RESPONSES_E2;'  ),
     ('DROP   TABLE TRM_LEAD_HISTORY.IMI_T7_SMS_RESPONSES_LOG;'),
     ('DROP   TABLE TRM_LEAD_HISTORY.IMI_T7_SMS_RESPONSES_TEST;'),
     ('CREATE SET TABLE TRM_LEAD_HISTORY.IMI_T7_SMS_RESPONSES_TEST,
          NO FALLBACK ,
          NO BEFORE JOURNAL,
          NO AFTER JOURNAL,
          CHECKSUM = DEFAULT,
          DEFAULT MERGEBLOCKRATIO
          (
              Response_Dttm           VARCHAR(100)
              ,From_Number            VARCHAR(20)
              ,To_Shortcode                VARCHAR(5)
              ,Response_Text            VARCHAR(100)
              ,Status                   VARCHAR(100)
              ,Folder                   VARCHAR(100)
              ,Folder_Owner             VARCHAR(100)
           );')   
           TO OPERATOR (DDL_OPERATOR);
);

Step Load_Table
(
   APPLY ('INSERT INTO TRM_LEAD_HISTORY.IMI_T7_SMS_RESPONSES_TEST
                                  (
                                  :Response_Dttm
                                  ,:From_Number
                                  ,:To_Shortcode
                                  ,:Response_Text
                                  ,:Status
                                  ,:Folder
                                  ,:Folder_Owner );')
   TO OPERATOR (LOAD_CSV)

   SELECT * FROM (OPERATOR READ_CSV);
);

);

it is giving error "syntax error at or near line 111 of job script file 'IMI_T7_SMS_RESPONSS.tpt: TPT_INFRA:At "Select" missing semicol_ in Rule:Step Compilation failed due to errors

For reference from code above line 111 is at the end of code "SELECT * FROM (OPERATOR READ_CSV);". I have tried everything but nothing seems to work. Can you kindly guide.

0

There are 0 best solutions below