Merge a large list of logical vectors

362 Views Asked by At

I have a large list of TRUE/FALSE logical vectors (144 list elements, each ~ 23 million elements long). I want to merge them using any to produce one logical vector. If any of the first elements of each list element are TRUE then TRUE is returned and so on for the length of the vectors. Here's an example:

#  Some data
set.seed(1)
ll <- replicate(3,sample(c(TRUE,FALSE),5,TRUE),simplify=F)

#[[1]]
#[1]  TRUE  TRUE FALSE FALSE  TRUE

#[[2]]
#[1] FALSE FALSE FALSE FALSE  TRUE

#[[3]]
#[1]  TRUE  TRUE FALSE  TRUE FALSE

#  What I want (and one way of doing it)...
apply( do.call(cbind,ll) , 1 , any )
#  [1]  TRUE  TRUE FALSE  TRUE  TRUE

Wait, you already posted a solution in that code, why ask the question?

I have 144 vectors, each of 23,721,703 length in my real data. Attempting the above throws errors such as:

# *** caught segfault ***
#address 0x18, cause 'memory not mapped'

OR

#Error in aperm.default(X, c(s.call, s.ans)) : 
#  long vectors not supported yet: memory.c:1648

I'm running R 3.0.2 on Ubuntu 64bit with 112Gb RAM.

1

There are 1 best solutions below

1
On BEST ANSWER

You can use Reduce

  Reduce('|', ll)

Benchmarks

set.seed(1)
ll <- replicate(144, sample(c(TRUE, FALSE), 1e5,
       replace=TRUE), simplify=FALSE)
system.time(apply(do.call(cbind, ll), 1, any))
# user  system elapsed 
# 0.575   0.022   0.598 

system.time(Reduce(`|`, ll))
# user  system elapsed 
# 0.287   0.008   0.295