MATLAB Database fetch() for WRDS Compustat taking too long

Question

I am connecting MATLAB to the Wharton Research Data Services COMPUSTAT database and am running a simple query

WRDS_Connect

setdbprefs('FetchInBatches','yes')
query = exec(WRDS, ['select gvkey from COMP.FUNDA where fyear = 2000 order by gvkey asc']);
result = fetch(query);
result.data

WRDS_Connect simply establishes a connection to WRDS server using JDBC drivers and

WRDS = database(instance, username, password, serverAddress);

The query is not too big and gives me 27000 rows of data. However it takes too long. Profiling shows that it takes about 5 minutes for these 27000 rows. I want to actually make more extensive queries but it should not take so long.

Is there anything I can do to speed things up? I've attached a screenshot of the profiling.

Thanks for your help!

Answer 1

If you have a PhD acount or above you can change approach a little bit.

I developed an API that lets you download datasets from WRDS unix servers. It's easy to use and faster.

Check it out on https://github.com/okomarov/wrds and let me know if you have problems.

wrds_install
w = wrds('username');
w.sas2csv('COMP.FUNDA');

The data will be saved in .\data\COMP.FUNDA.zip.

You can then:

1. unzip the table
2. import it with readtable()
3. apply selction/filters in Matlab