How do I get the data from a webpage into iseries and work with it using Python

Question

I have a list of dr saves details made availaible on our intranet page.

I need to build a utility such thatthe input dates provided ,somehow gets the volume and seq number details from the webpage and access the db2 database accordingly.

I was thinking/learning pyhon on terms of using bs4 soup to scrape the data on webpage but now how do I get the data into iseries. Is this even possible ?

Answer 1

you can use the sql function httpgetclob to read the contents of a web page.

Here is sample code that uses httpgetclob in an sql procedure. The procedure receives the httpgetclob output into a clob variable. Then it reads 256 byte chunks of that output, writing the chunks to an output file.

here is the output file:

/* test0244 - web page text contents.        */           
                                                          
CREATE OR REPLACE TABLE TEST0244 (
pageName   char(20),
seqnbr   decimal(7,0),
text     char(256) not null default ' ', 
constraint test0244_pk primary key( pageName, seqnbr )    
) ; 

sql procedure that gets the contents of the web page and writes to the output table:

/* test0244_getWebPage - get contents of web page.    */ 
/*                       Store in table TEST0244.     */ 

CREATE or replace PROCEDURE test0244_getWebPage( 
 in inPageName char(20), 
 in inUrl    char(256) 
 ) 
  LANGUAGE           SQL
  MODIFIES           SQL DATA
  SET OPTION         COMMIT = *NONE 
BEGIN
declare     vSqlCode decimal(5,0) ; 
declare     sqlCode int DEFAULT 0 ;                      
declare      url varchar(2000) ;                         
declare      webClob clob(500000) ;                      
declare      web_lx int ;                                
declare      bx  int ;                                   
declare      rem int default 0 ; 
declare      segSx int ; 
declare      segLx int ; 
declare      segText varchar(2000) ; 
declare      seqnbr decimal(7,0) default 0 ; 

/* clear current page contents.      */               
delete from  test0244 a                               
where        a.pageName = inPageName ;                
                                                      
set        url = inUrl ;                              
set        webClob = SYSTOOLS.httpGetClob( url, '' ) ;
set        web_lx  = length( webClob ) ;              
set        bx = 1 ;                                   
set        segSx = 256 ;                              
set        seqnbr = 0 ;                               
while( bx <= web_lx ) do                              
set        rem = web_lx - bx + 1 ;                    
set        segLx = case when rem >= segSx             
                        then segSx                    
                        else rem end ;                
set        segText = substr( webClob, bx, segLx ) ;   
set        seqnbr  = seqnbr + 1 ;                     
insert into   test0244  ( pageName, seqnbr, text )    
values( inPageName, seqnbr, segText ) ;               
                                                      
set        bx = bx + segLx ;                          
end while ;
END 

code that calls the procedure:

call   test0244_getWebPage ( 'craigslist      ',   
          'https://newjersey.craigslist.org/')