Get same completion candidates for IMain as would appear in a REPL

Question

I am trying to get the code completion for a Scala interpreter to work. Ideally it would work identically to the one provided by the REPL (ILoop). I am using a text document as source, so I do not want to instantiate an ILoop but just IMain.

In the following example, the completion only works for special cases:

import scala.tools.nsc.interpreter.{JLineCompletion, IMain}
import scala.tools.nsc.Settings

object CompletionTest extends App {
  val settings  = new Settings
  settings.usejavacp.tryToSetFromPropertyValue("true")
  val intp      = new IMain(settings)
  intp.initializeSynchronous()
  assert(intp.isInitializeComplete)
  val comp      = new JLineCompletion(intp)
  val completer = comp.completer()
  val buffer    = "val x = Indexe"
  val choices   = completer.complete(buffer, buffer.length)
  println("----BEGIN COMPLETION----")
  choices.candidates.foreach(println)
  println("----END COMPLETION----")
  intp.close()
}

The expected output would be IndexedSeq, but it is empty. If I set the buffer to just Indexe, it works. If I set the buffer to   Indexe (leading whitespace), the completion candidates are empty again.

So there must be an additional step involved in processing the buffer or invoking the completion. What exactly happens when <tab> is pressed in the REPL? It seems almost impossible to figure out which methods are called...

Answer 1

In JLineReader, you can see the wiring. JLineConsoleReader sets up an ArgumentCompleter with the ScalaCompleter as the underlying completer.

So the completer wants just the argument, not the line.

apm@mara:~$ scalam
Welcome to Scala version 2.11.0-M7 (OpenJDK 64-Bit Server VM, Java 1.7.0_25).
Type in expressions to have them evaluated.
Type :help for more information.

scala> :power
** Power User mode enabled - BEEP WHIR GYVE **
** :phase has been set to 'typer'.          **
** scala.tools.nsc._ has been imported      **
** global._, definitions._ also imported    **
** Try  :help, :vals, power.<tab>           **

scala> val b = "Indexe"
b: String = Indexe

scala> completion.completer complete (b, b.length)
res0: scala.tools.nsc.interpreter.Completion.Candidates = Candidates(0,List(IndexedSeq))

In other keystrokes,

// paste your code

scala> val buffer    = "Indexe"
buffer: String = Indexe

scala> completer.complete(buffer, buffer.length)
res6: scala.tools.nsc.interpreter.Completion.Candidates = Candidates(0,List(IndexedSeq))

scala> import tools.nsc.interpreter.Completion.Candidates
import tools.nsc.interpreter.Completion.Candidates

scala> val Candidates(_, choices) = completer.complete(buffer, buffer.length)
choices: List[String] = List(IndexedSeq)

scala> choices foreach println
IndexedSeq

To hand it the full line:

scala> val argCompletor: ArgumentCompleter =new ArgumentCompleter(new JLineDelimiter, scalaToJline(comp.completer))
argCompletor: jline.console.completer.ArgumentCompleter = jline.console.completer.ArgumentCompleter@751222c7

scala> val maybes = new java.util.ArrayList[CharSequence]
maybes: java.util.ArrayList[CharSequence] = []

scala> val buffer    = "val x = Indexe"
buffer: String = val x = Indexe

scala> argCompletor.setStrict(false)

scala> argCompletor.complete(buffer, buffer.length, maybes)
res32: Int = 8

scala> maybes
res33: java.util.ArrayList[CharSequence] = [IndexedSeq]

The delimiter does the line parse.

Edit - some value-added analysis:

"Strict" mode for the completor exists because you can supply a completor for each token on the line, and require each previous argument to be completable. For n completors, all args after nth arg are handled by the last completor.

Answer 2

Partial answer. I managed to drill a hole in that monster by overriding scalaToJline in JLineReader. That method is invoked with a pre-massaged string, following this trace:

at CompletionTest$$anon$1$$anon$2$$anon$3.complete(CompletionTest.scala:37)
at scala.tools.jline.console.completer.ArgumentCompleter.complete(ArgumentCompleter.java:150)
at scala.tools.jline.console.ConsoleReader.complete(ConsoleReader.java:1543)
at scala.tools.jline.console.ConsoleReader.readLine(ConsoleReader.java:1312)
at scala.tools.jline.console.ConsoleReader.readLine(ConsoleReader.java:1170)
at scala.tools.nsc.interpreter.JLineReader.readOneLine(JLineReader.scala:74)
at scala.tools.nsc.interpreter.InteractiveReader$$anonfun$readLine$2.apply(InteractiveReader.scala:42)
at scala.tools.nsc.interpreter.InteractiveReader$$anonfun$readLine$2.apply(InteractiveReader.scala:42)
at scala.tools.nsc.interpreter.InteractiveReader$.restartSysCalls(InteractiveReader.scala:49)
at scala.tools.nsc.interpreter.InteractiveReader$class.readLine(InteractiveReader.scala:42)
at scala.tools.nsc.interpreter.JLineReader.readLine(JLineReader.scala:19)
at scala.tools.nsc.interpreter.ILoop.readOneLine$1(ILoop.scala:568)
at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:584)
at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:587)

Answer 3

So this my "unwiring" which seems to be working as expected:

import scala.tools.nsc.interpreter._
import scala.tools.jline.console.completer.{Completer, ArgumentCompleter}
import scala.tools.nsc.interpreter.Completion.{Candidates, ScalaCompleter}
import scala.tools.nsc.Settings
import collection.JavaConverters._

 

object Completion2 extends App {
  val settings  = new Settings
  settings.usejavacp.tryToSetFromPropertyValue("true")
  val intp      = new IMain(settings)
  intp.initializeSynchronous()

  val completion = new JLineCompletion(intp)

  def scalaToJline(tc: ScalaCompleter): Completer = new Completer {
    def complete(_buf: String, cursor: Int, candidates: JList[CharSequence]): Int = {
      val buf   = if (_buf == null) "" else _buf
      val Candidates(newCursor, newCandidates) = tc.complete(buf, cursor)
      newCandidates foreach (candidates add _)
      newCursor
    }
  }

  val argCompletor: ArgumentCompleter =
    new ArgumentCompleter(new JLineDelimiter, scalaToJline(completion.completer()))
  argCompletor.setStrict(false)

  val jlist: java.util.List[CharSequence] = new java.util.ArrayList

  val buffer = "val x = Indexe"

  argCompletor.complete(buffer, buffer.length, jlist)

  val list = jlist.asScala

  println("----BEGIN COMPLETION----")
  list.foreach(println)
  println("----END COMPLETION----")
  intp.close()
}

---

Edit: This has problems with wildcard imports for some reason. Like if I execute

import mypackage.MySymbol

Then MySymbol is found by the completer. But if I execute instead

import mypackage._

Then none of the contents of mypackage are found. Any ideas?