download.file() "cannot open URL"

Question

I'm trying to figure out what's wrong with this URL or why my RStudio can't access a perfectly fine URL I can reach with my browser.
This URL automatically downloads a .csv The code has worked for colleagues in the past.

url <- "http://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=6329&Year=1953&Month=1&Day=1&timeframe=2&submit= Download+Data"

download.file(url,"files.csv",quiet=TRUE)

The error it's throwing is:

Error in download.file(url, "files.csv", quiet = TRUE) : 
  cannot open URL 'http://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=6329&Year=1953&Month=1&Day=1&timeframe=2&submit= Download+Data'

In addition: Warning message:

In download.file(url, "files.csv", quiet = TRUE) :
  URL 'http://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=6329&Year=1953&Month=1&Day=1&timeframe=2&submit= Download+Data': status was 'URL using bad/illegal format or missing URL'

Answer 1

so from inspecting your code, my best guess is that the problem is likely because there is whitespace in the link you assigned to the "url" variable, specifically before "Download+Data." I was able to replicate and solve the problem by removing the whitespace from the link and downloading the file like this (do note that I printed the dataframe to show the file was successlly downloaded:

> # Remove whitespace
> url <- "http://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=6329&Year=1953&Month=1&Day=1&timeframe=2&submit=Download+Data"
> download.file(url, "files.csv", quiet = T)
> 
> df <- read.csv("files.csv")
> 
> head(df)
  Longitude..x. Latitude..y. Station.Name Climate.ID  Date.Time Year Month Day Data.Quality
1        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-01 1953     1   1           NA
2        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-02 1953     1   2           NA
3        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-03 1953     1   3           NA
4        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-04 1953     1   4           NA
5        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-05 1953     1   5           NA
6        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-06 1953     1   6           NA
  Max.Temp...C. Max.Temp.Flag Min.Temp...C. Min.Temp.Flag Mean.Temp...C. Mean.Temp.Flag
1          -5.6            NA         -16.7            NA          -11.2             NA
2          -1.1            NA         -13.3            NA           -7.2             NA
3           4.4            NA         -13.3            NA           -4.5             NA
4           8.3            NA           1.7            NA            5.0             NA
5           3.3            NA          -3.9            NA           -0.3             NA
6           1.1            NA         -11.1            NA           -5.0             NA
  Heat.Deg.Days...C. Heat.Deg.Days.Flag Cool.Deg.Days...C. Cool.Deg.Days.Flag Total.Rain..mm.
1               29.2                 NA                  0                 NA             0.0
2               25.2                 NA                  0                 NA             0.0
3               22.5                 NA                  0                 NA            63.5
4               13.0                 NA                  0                 NA             0.0
5               18.3                 NA                  0                 NA             0.0
6               23.0                 NA                  0                 NA             0.0
  Total.Rain.Flag Total.Snow..cm. Total.Snow.Flag Total.Precip..mm. Total.Precip.Flag
1                               0                               0.0                  
2                               0                               0.0                  
3                               0                              63.5                  
4                               0                               0.0                  
5                               0                               0.0                  
6                               0                               0.0                  
  Snow.on.Grnd..cm. Snow.on.Grnd.Flag Dir.of.Max.Gust..10s.deg. Dir.of.Max.Gust.Flag
1                NA                NA                        NA                   NA
2                NA                NA                        NA                   NA
3                NA                NA                        NA                   NA
4                NA                NA                        NA                   NA
5                NA                NA                        NA                   NA
6                NA                NA                        NA                   NA
  Spd.of.Max.Gust..km.h. Spd.of.Max.Gust.Flag
1                     NA                   NA
2                     NA                   NA
3                     NA                   NA
4                     NA                   NA
5                     NA                   NA
6                     NA                   NA

Answer 2

Remove submit= part from URL.

> url<- "http://climate.weather.gc.ca/climate_data/bulk_data_e.html?format=csv&stationID=6329&Year=1953&Month=1&Day=1&timeframe=2"
> tmp <- tempfile()
> download.file(url, tmp, quiet=TRUE)
> read.csv(tmp)[1:3, 1:10]
  Longitude..x. Latitude..y. Station.Name Climate.ID  Date.Time Year Month Day Data.Quality Max.Temp...C.
1        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-01 1953     1   1           NA          -5.6
2        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-02 1953     1   2           NA          -1.1
3        -62.02        45.48 COLLEGEVILLE    8201000 1953-01-03 1953     1   3           NA           4.4
> 
> unlink(tmp)