How to escape characters in a ftp url

Question

I am using spark for fetching a file from ftp server using the following code

val dataSource = "ftp://<username>:<password>@<host>:<port>/<path>"
sc.addFile(dataSource)
var fileName = SparkFiles.get(dataSource.split("/").last)
val sqlContext = new SQLContext(sc)
val df = sqlContext.read
  .format("com.databricks.spark.csv")
  .option("header", "true") // Use first line of all files as header
  .option("inferSchema", "true") // Automatically infer data types
  .option("delimiter", ",")
  .load(fileName)

but i am facing the problem of escaping characters like '^', '!', '@' or '#' etc etc. I tried using the HEX ASCII values for the special characters and tried to escape them by surrounding the username/password in escape quotes for example

val dataSource = "ftp://\'<username>\':\'<password>\'.........

but this didn't work either. Can anyone please explain to me how I can escape such characters in my ftp url ?

P.S., this code does work when the password doesn't contain any special characters.

Answer 1

Remember that it's a URL. That means it can include % escapes. For example an anonymous ftp site requires an email address as password, so it would look like

ftp://anonymous:user%[email protected]/path/to/file