I have a rust application that uses "azure_storage_datalake" at https://github.com/ Azure/Azure-Sdk-for-rust. The application can enumerate all of the files, and I can determine the name and size of each file. What I haven't been able to figure out is how to get the Content-MD5 for each file. I hope to get the value rather than having to scan each file and determine the MD5 values on the fly. The RESTful API that Azure provides shows the Content-MD5 can be determined,
I'd appreciate any tips on how to get it using the "azure_storage_datalake" sdk.
Thank you for your time and interest.
Here is the code that I have been developing:
let data_lake_client = DataLakeClient::new(
self.account_name.to_owned(),
self.storage_credentials.to_owned(),
);
let file_system_client = data_lake_client.file_system_client(container_name);
let directory_client = file_system_client.clone().into_directory_client(".");
let mut page_cnt = 0;
let mut cnt = 0;
let mut response = directory_client
.list_paths()
.directory("alm")
.recursive(true)
.max_results(NonZeroU32::new(paging_size).unwrap())
.into_stream()
.enumerate();
loop {
let page = response.next().await;
match page {
Some((_, p)) => match p {
Ok(_) => {
page_cnt += 1;
let paths = p.unwrap().paths;
cnt += paths.len();
paths.iter().for_each(|file| {
if !file.is_directory {
println!("[{}/{}]: {}, Size: {} bytes, MD5: '{}'",
page_cnt,
cnt,
file.name,
file.content_length as u64,
"tbd".to_string(), // <<<<< How to get the ContentMD5 value for this file?
);
}
});
}
Err(e) => {
println!("{:#?}", e);
break;
}
},
None => {
break;
}
}
}
The following is new code that replaces the "if !file.is_directory" block. It doesn't produce an error, but the properties for any file are always empty.
if !file.is_directory {
let path_name = file.name.clone();
let file_client = file_system_client.get_file_client(&path_name);
let file_properties = file_client.get_properties().await;
println!("{:?}", file_properties.unwrap().properties.unwrap().get("CONTENT-MD5").unwrap());
}
Within your
if b.is_directorybranch, you can create aFileClientinstance by callingfile_system_client.get_file_clientmethod. Then use theget_propertiesmethod to access the properties associated with the file. This will contain the MD5 details you are looking for.Note: This solution is not tested.