Accessing data on remote machine's clipboard when running tests using selenium-webdriver

Question

I have a setup as follows:

Machine 1 - Ubuntu - selenium hub
Machine 2 - Ubuntu - selenium node
Machine 3 - Windows 8.1 - selenium node

To execute tests i am using RemoteWebDriver and configured Jenkins CI on hub.

Issue: URL i am testing has a button that copies data to clipboard on clicking it. The data is not visible to end user unless pasted. My concern is when running locally, i am able to get system clipboard data using:

strScript = (String) Toolkit.getDefaultToolkit().getSystemClipboard()
        .getData(DataFlavor.stringFlavor);

But when triggering code from jenkins on remote machines I am unable to fetch the contents and receiving below error:

java.awt.HeadlessException: 
No X11 DISPLAY variable was set, but this program performed an operation which requires it.

Its clear that it is trying to fetch data on jenkins server, but certain display issue. But my requirement is to get the data that got stored on the remote machine after clicking the button.

Any solutions?

Thanks.

Answer 1

If the Selenium Grid environment is in your control and you have the luxury of setting up the Nodes, then here's how you go about building this.

• Build a custom node servlet which when invoked would return back the contents of the clip board.
• Start the node with this newly built custom servlet.
• Use a library such as Talk2Grid to find the IP and port of the node to which your session was routed to.
• use the IP and port of obtained, to invoke the custom servlet to get access to the clipboard.

The below sample should explain all of this in action (I am using selenium 3.141.59)

Dependencies being used

<dependency>
  <groupId>org.seleniumhq.selenium</groupId>
  <artifactId>selenium-java</artifactId>
  <version>3.141.59</version>
</dependency>
<dependency>
  <groupId>org.seleniumhq.selenium</groupId>
  <artifactId>selenium-server</artifactId>
  <version>3.141.59</version>
</dependency>
<dependency>
  <groupId>com.rationaleemotions</groupId>
  <artifactId>talk2grid</artifactId>
  <version>1.2.1</version>
</dependency>
<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>7.0.0</version>
</dependency>

Custom servlet looks like below

package com.rationaleemotions.servlets;

import java.awt.Toolkit;
import java.awt.datatransfer.DataFlavor;
import java.awt.datatransfer.UnsupportedFlavorException;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.openqa.selenium.json.Json;

public class ClipboardServlet extends HttpServlet {

  @Override
  protected void doGet(HttpServletRequest request, HttpServletResponse response)
      throws IOException {
    response.setContentType("application/json");
    response.setCharacterEncoding("UTF-8");
    response.setStatus(200);
    try {
      String text = Toolkit.getDefaultToolkit().getSystemClipboard()
          .getData(DataFlavor.stringFlavor)
          .toString();
      Map<String, String> map = new HashMap<>();
      map.put("contents", text);
      response.getWriter().append(new Json().toJson(map));
    } catch (UnsupportedFlavorException e) {
      e.printStackTrace();
    }
  }
}

Here lets assume that our servlet belongs to the jar named "servlets.jar". Here's how you start off the node

java -cp servlets:selenium-server-standalone-3.141.59.jar org.openqa.grid.selenium.GridLauncherV3 -role node -servlets com.rationaleemotions.servlets.ClipboardServlet

The hub is started using the command

java -jar selenium-server-standalone-3.141.59.jar -role hub

Here's a sample that is going to use all of this.

package com.rationaleemotions;

import com.rationaleemotions.pojos.Host;
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import okhttp3.Call;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.By;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.json.Json;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

public class ClipboardSample {

  private RemoteWebDriver driver;

  @BeforeClass
  public void beforeClass() throws MalformedURLException {
    driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"),
        DesiredCapabilities.chrome());
  }

  @Test
  public void demo() throws Exception {
    String text = "Mussum ipsum cacilds, vidis litro abertis. Consetis adipiscings elitis. Pra lá , depois divoltis porris, paradis. Paisis, filhis, espiritis santis. Mé faiz elementum girarzis, nisi eros vermeio, in elementis mé pra quem é amistosis quis leo. Manduma pindureta quium dia nois paga.";
    driver.navigate().to("https://googlechrome.github.io/samples/async-clipboard/");

    driver.findElement(By.cssSelector("textarea#out")).sendKeys(text);
    TimeUnit.SECONDS.sleep(1);
    takeScreenshot();
    driver.findElement(By.cssSelector("button#copy")).click();
    TimeUnit.SECONDS.sleep(1);
    Host grid = new Host("localhost", "4444");
    Host node = new GridApiAssistant(grid)
        .getNodeDetailsForSession(driver.getSessionId().toString());
    String url = String
        .format("http://%s:%d/extra/ClipboardServlet", node.getIpAddress(), node.getPort());
    Request request = new Request.Builder().url(url).build();

    Call call = new OkHttpClient.Builder().build().newCall(request);
    Response response = call.execute();
    String localClipboardData = response.body().string();
    Map<String, String> data = new Json().toType(localClipboardData, Map.class);
    Assert.assertEquals(text, data.get("contents"));
  }

  private void takeScreenshot() throws IOException {
    File src = driver.getScreenshotAs(OutputType.FILE);
    File target = new File(
        System.getProperty("user.dir") + File.separator + "target" + File.separator +
            "screenshot.jpg");
    FileUtils.copyFile(src, target);
  }

  @AfterClass
  public void afterClass() {
    if (driver != null) {
      driver.quit();
    }
  }
}