blank.lines.skip = TRUE fails with read.fwf?

Question

There are four blank lines at the end of my file.

> data=read.fwf("test2",head=F,widths=c(3,1,-3,4,-1,4),blank.lines.skip = TRUE)  
> data  

When I run this code, the blank.lines.skip argument is ignored. I still get blank lines in my output.

The file is:

x1     F          1890 1962  
x2                1857 1936  
x3                1900 1978  
x4                1902 1994  
x5        F       1878 1939 

and four blank lines at the end.

Answer 1

It looks like you're right that blank.lines.skip does not apply to read.fwf -- would have to dig in the code to figure out why, but read.fwf does significant processing of the file before passing it (along with the blank.lines.skip directive) to read.table. However, it's not too hard to detect and remove all-blank lines after the fact.

For example:

cat("abc","def","ghi","","","",sep="\n",file="test3.dat")
read.table("test3.dat")  ## blank lines skipped (by default)
(x <- read.fwf("test3.dat",widths=c(1,1,1),blank.lines.skip=TRUE))
##     V1   V2   V3
## 1    a    b    c
## 2    d    e    f
## 3    g    h    i
## 4 <NA> <NA> <NA>
## 5 <NA> <NA> <NA>
## 6 <NA> <NA> <NA>
all_NA <- apply(x,1,function(z) all(is.na(z)))
x[!all_NA,]

To answer your other question (which you posted as an answer, which was then deleted by a moderator; you should generally add necessary modifications to your question by editing your original post, or by commenting on questions if necessary, rather than posting an answer): colClasses is indeed not "clever" enough for you to use automatic detection on most columns but override it for (a) specific column(s).