Scriptella: copy table from access to postgresql

Question

I'm new to Scriptella, and I got totally stuck with the following problem.

I have the access table given. I'm getting the table schema using program I wrote. The output file (tableschema.xml) looks like :

<?xml version="1.0" encoding="UTF-8"?>
<table>
    <column>ID</column>
    <datatype>COUNTER</datatype>
    <column>FirstName</column>
    <datatype>VARCHAR</datatype>
    <column>LastName</column>
    <datatype>VARCHAR</datatype>
    <column>Salary</column>
    <datatype>CURRENCY</datatype>
    <column>SSN</column>
    <datatype>INTEGER</datatype>
</table>

Then, using Scriptella, I need to create a new PostgreSQL Database (if possible. If not possible, we can assume, that DB is already created). Than i need to create a new table (required) using the XML file provided and copy all data from access table to PostgreSQL table.

I implemented data extraction from access. I implemented info extraction from XML file.

I got stuck with creating the DB and table in Postgres. It seems that CREATE DATABASE and CREATE TABLE just don't work through Scriptella.

My rough draft of etl.xml file:

<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
    <connection id="db1" url="jdbc:odbc:TestDB" driver="scriptella.driver.odbc.Driver" user="Dzmitry" password="a"/>

    <connection id="db2" url="jdbc:odbc:PostgreSQLconnector" driver="scriptella.driver.odbc.Driver" classpath="postgresql.jar" user="Dzmitry" password="a"/>

    <connection id="script" driver="script"/>
    <connection id="java" driver="scriptella.driver.janino.Driver"/>
    <connection id="log" driver="text"/>

    <!--  doesn't work

    <script connection-id="db2" new-tx="true" if="create_databases">
            CREATE DATABASE testDB;
    </script>


    -->


    <query connection-id="db1">
        SELECT * FROM GenInfo;

        <script connection-id="script">
            java.lang.System.out.println("Processing row number " + rownum + " ");
        </script>
        <script connection-id="log">
            ${FirstName}, ${LastName}, ${Salary}, ${SSN}
        </script>
    </query>

    <query connection-id="java">
        import javax.xml.parsers.*;
        import javax.xml.xpath.*;
        import java.io.*;
        import org.w3c.dom.Document;
        import org.xml.sax.*;

        DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
        DocumentBuilder docBuilder = dbFactory.newDocumentBuilder();

        File file = new File("tableschema.xml");
        Document document = docBuilder.parse(file);

        XPathFactory xpFactory = XPathFactory.newInstance();
        XPath xPath = xpFactory.newXPath();

        int numberOfColumns = Integer.parseInt(xPath.evaluate("count(/table/column)", document));


        for(int i = 1; i &lt;= numberOfColumns; i++){ // does not take less than sign
            System.out.print("Column name " + i + ": " + xPath.evaluate("/table/column[" + i + "]", document));
            System.out.println("\tData type " + i + ": " + xPath.evaluate("/table/datatype[" + i + "]", document));
        }

    </query>
</etl>

Could anybody help me?
Regards, Dzmitry.

Answer 1

In your script you are using an ODBC driver to access Postgres:

driver="scriptella.driver.odbc.Driver"

I suggest to use a JDBC driver as explained in the document :

<connection driver="postgresql" url="jdbc:postgresql://localhost:5432/DATABASENAME" user="username" password="password">
</connection>

Also, you can omit new-tx="true" attribute, I don't see a point of using it for your scenario. So start with something simple like this:

<connection connection-id="db2" driver="postgresql" url="jdbc:postgresql://localhost:5432/DATABASENAME" user="Dzmitry" password="a" classpath="postgresql.jar">
</connection>

<script connection-id="db2">
    CREATE TABLE distributors (
        did     integer PRIMARY KEY,
        name    varchar(40)
    );
</script>

In this example I showed how to create a table for an existing database DATABASENAME. If you want to create another database, the script will be more complicated, but before I provide additional details, I need you to confirm that you are now able to create tables in the existing database.

In order to create a database, first you have to connect to an existing one like template1(it should come preinstalled). And create another database from there. Then execute other scripts using a separate connection. Please note that the connection for create table scripts have to enable lazy-init attribute, otherwise the error will be thrown for unexisting database:

<!-- Connection for just for creating a database -->
<connection connection-id="db2_init" driver="postgresql" url="jdbc:postgresql://localhost:5432/template1" user="postgres" password="postgres" classpath="postgresql.jar">
</connection>

<!-- Connection for DDL and data statements. Needs to be lazy-init=true -->
<connection connection-id="db2_tables" driver="postgresql" lazy-init="true" url="jdbc:postgresql://localhost:5432/testDB" user="Dzmitry" password="a" classpath="postgresql.jar">
</connection>

<script connection-id="db2_init" if="create_databases">
    CREATE DATABASE testDB;
</script>
<script connection-id="db2_tables" if="create_schema">
    CREATE TABLE distributors (
        did     integer PRIMARY KEY,
        name    varchar(40)
    );
</script>