I am trying to download entire metadata from Europe PMC dataset. I am able to download the metadata through rest api provided by the site. But the constraint is the volume of the data and because of that I am facing some performance issues.
Recently i have downloaded around 20,000 records with 47000 variables. Europe PMC has approx 31 million such records.
Is there any way I can download Europe PMC dataset using FTP or any more effcient way? FTP download for metadata is available for only open access articles as of now
You can download the metadata of all full text articles on Europe PMC via FTP.
You can find all of the FTP downloads available here: https://europepmc.org/downloads
The one that contains the metadata is 'Metadata of all Full-Text Europe PMC articles' and the FTP is here: https://europepmc.org/ftp/pmclitemetadata/
Then you could use a flag like HAS_FT:N to access the metadata for all other articles via the API. It's still a lot of articles, though.