I am trying to replace the column "X" of dataframe df1 by column "X" of dataframe df2. I use the below code:
df1$X <- df2$X
The above line of code works well in RStudio environment. However, when I run the R script on my Ubuntu VM from command line and load the dataframes by CSV files fetched from hdfs, the above line of code gives the below error:
Error in
$<-.data.frame(*tmp*, "X", value = c(3.688879454, :
replacement has 1464 rows, data has 161
I tried different indexing methods and solutions from google but could not solve it. I am guessing the issue might be because column "X" of df1 has some missing values. I cannot find a solution to fix this.
Here is the code prior to the line of code in question:
require(rhdfs)
hdfs.init()
f = hdfs.file(file_input,"r",buffersize=104857600)
m = hdfs.read(f)
con = rawToChar(m)
Df1 = read.table(textConnection(con), sep = ",",header=TRUE, fill = TRUE)
f1 = hdfs.file(file_input1,"r",buffersize=104857600)
m1 = hdfs.read(f1)
con1 = rawToChar(m1)
df2 = read.table(textConnection(con1), sep = ",",header=TRUE, fill = TRUE)
Df1$X <- df2$X