TT#17853 Fix selenium related toolchain, replacing selenium with geckodriver

An unintended docker image rebuild of
ngcp-panel-tests-selenium-jessie resulted in failures with
our selenium tests. A series of problems started:

1) Updating the Selenium version resulted in an error message by
Selenium, requesting the geckodriver binary
(https://github.com/mozilla/geckodriver - being a proxy for using
W3C WebDriver-compatible clients to interact with Gecko-based
browsers). So Selenium versions >=3 depend on geckodriver.

2) Recent versions of Selenium also depend on Java >=8, available
only via backports for jessie. Since we avoid usage of backports
as much as possible and we'd have to upgrade once anyway we
decided to base our fresh docker image on stretch.

3) We have to backport Selenium::Remote::Driver (Debian package
libselenium-remote-driver-perl) for stretch (since it's not
available in Debian, even though there's an ITP:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839569). It has
a bug which prevents its usage for us:
https://github.com/gempesaw/Selenium-Remote-Driver/issues/319 ->
We fixed the problem via our own backport and submitted a PR
towards upstream:
https://github.com/gempesaw/Selenium-Remote-Driver/pull/321

4) Firefox from Debian/stretch (firefox-esr 52.2.0esr-1~deb9u1)
fails with geckodriver e.g. in click events, the current Firefox
version 54 works fine though. So instead install only the
relevant runtime packages from Debian and use the Firefox version
v54 from upstream.

Resulting changes:

* switch from Debian/jessie to Debian/stretch
* no longer use selenium at all but only geckodriver instead
  (recent versions of selenium depend on geckodriver anyway)
* do all package/file changes in /home/selenium/ (to clearly
  see what's from us/docker and what are actual temporary files)
* testrunner related changes:
  - invoke vncserver with `-localhost no` option to accept
    VNC connections not only from localhost only
  - adjust vncserver process name for Xvnc4->Xtigervnc switch
  - start geckodriver instead of selenium
* package related changes:
  - backported libtest-pretty-perl (v0.32)
  - backported libselenium-remote-driver-perl (v1.20)
  - add curl to package list since it's handy for debugging test
    failures, like e.g. `curl -X POST localhost:4444/session --data {}`
  - add procps to have pkill binary available inside testrunner
  - openjdk/java is no longer needed since we don't depend on selenium anymore
  - vnc4server is now a transitional package for
    tigervnc-standalone-server, so directly depend on it

Change-Id: I607ffd4fce80f9e84b76c3c7a19824819ac4503c

t/selenium/Dockerfile

@@ -1,46 +1,57 @@
-# DOCKER_NAME=ngcp-panel-tests-selenium-jessie
-FROM docker.mgm.sipwise.com/sipwise-jessie:latest
+# DOCKER_NAME=ngcp-panel-tests-selenium-stretch
+FROM docker.mgm.sipwise.com/sipwise-stretch:latest
 
 # Important! Update this no-op ENV variable when this Dockerfile
 # is updated with the current date. It will force refresh of all
 # of the base images and things like `apt-get update` won't be using
 # old cached versions when the Dockerfile is built.
-ENV REFRESHED_AT 2017-06-21
+ENV REFRESHED_AT 2017-06-23
 
 RUN apt-get update
 RUN apt-get install --assume-yes \
-	devscripts \
+        curl \
+        devscripts \
         fluxbox \
         git \
-        iceweasel \
         libtap-formatter-junit-perl \
         lsof \
         net-tools \
-        openjdk-7-jre-headless \
+        procps \
         vim-tiny \
-        vnc4server \
+        tigervnc-standalone-server \
         wget \
         xterm
 
+# NOTE: we used to use selenium for testing, nowadays we're using
+# only firefox with geckodriver and libselenium-remote-driver-perl
+# without the need for selenium
 RUN adduser --disabled-password --gecos=Selenium-User selenium
 
-WORKDIR /home/selenium
-RUN wget -O selenium-server-standalone-2.47.1.jar http://selenium-release.storage.googleapis.com/2.47/selenium-server-standalone-2.47.1.jar
-RUN ln -s selenium-server-standalone-2.47.1.jar selenium-server-standalone.jar
+WORKDIR /home/selenium/
 
-WORKDIR /tmp/
-COPY t/selenium/control /tmp/
-RUN mk-build-deps /tmp/control
-RUN dpkg --force-depends -i /tmp/ngcp-panel-tests-selenium-build-deps_1.0_all.deb
+# build-dependencies
+COPY t/selenium/control /home/selenium/control
+RUN mk-build-deps /home/selenium/control
+RUN dpkg --force-depends -i /home/selenium/ngcp-panel-tests-selenium-build-deps_1.0_all.deb
 RUN apt-get --assume-yes -f install
 
-RUN wget http://deb.sipwise.com/files/libselenium-remote-driver-perl_0.26-1~bpo80+1_all.deb
-RUN dpkg --force-depends -i libselenium-remote-driver-perl_0.26-1~bpo80+1_all.deb
+# the connection between firefox and our selenium tests
+RUN wget https://deb.sipwise.com/files/libselenium-remote-driver-perl_1.20-0.1~bpo9+1sipwise1_all.deb
+RUN dpkg --force-depends -i libselenium-remote-driver-perl_1.20-0.1~bpo9+1sipwise1_all.deb
 RUN apt-get --assume-yes -f install
+RUN wget https://deb.sipwise.com/files/geckodriver-v0.17.0-linux64.tar.gz
+RUN tar xvf geckodriver-v0.17.0-linux64.tar.gz
+
+# firefox-esr 52.2.0esr-1~deb9u1 from Debian/stretch fails with geckodriver e.g. in click events
+RUN wget https://deb.sipwise.com/files/firefox-54.0.tar.bz2
+RUN tar xvf firefox-54.0.tar.bz2
+RUN ln -s /home/selenium/firefox/firefox /usr/bin/firefox
+# runtime dependencies for firefox
+RUN apt-get --assume-yes install libgtk-3-0 libdbus-glib-1-2
 
 # for prove execution by developers
-RUN wget http://deb.sipwise.com/files/libtest-pretty-perl_0.31-1~bpo80+1_all.deb
-RUN dpkg --force-depends -i libtest-pretty-perl_0.31-1~bpo80+1_all.deb
+RUN wget https://deb.sipwise.com/files/libtest-pretty-perl_0.32-0.1~bpo9+1_all.deb
+RUN dpkg --force-depends -i libtest-pretty-perl_0.32-0.1~bpo9+1_all.deb
 RUN apt-get --assume-yes -f install
 
 RUN echo "cd /code && ./t/selenium/testrunner 1.2.3.4" >/home/selenium/.bash_history
@@ -49,24 +60,22 @@ EXPOSE 5999
 
 COPY t/selenium/jenkins_docker_run /home/selenium/
 
-WORKDIR /home/selenium
-
 ################################################################################
 # Instructions for usage
 # ----------------------
 # When you want to build the base image from scratch (jump to the next section if you don't want to build yourself!):
 # NOTE: run the following command from root folder of git repository:
-# % docker build --tag="ngcp-panel-tests-selenium-jessie" -f t/selenium/Dockerfile .
-# % docker run --rm -p 5999:5999 -i -t -v $(pwd):/code:rw ngcp-panel-tests-selenium-jessie:latest bash
+# % docker build --tag="ngcp-panel-tests-selenium-stretch" -f t/selenium/Dockerfile .
+# % docker run --rm -p 5999:5999 -i -t -v $(pwd):/code:rw ngcp-panel-tests-selenium-stretch:latest bash
 #
 # Use the existing docker image:
-# % docker pull docker.mgm.sipwise.com/ngcp-panel-tests-selenium-jessie
-# % docker run --rm -p 5999:5999 -i -t -v $(pwd):/code:rw docker.mgm.sipwise.com/ngcp-panel-tests-selenium-jessie:latest bash
+# % docker pull docker.mgm.sipwise.com/ngcp-panel-tests-selenium-stretch
+# % docker run --rm -p 5999:5999 -i -t -v $(pwd):/code:rw docker.mgm.sipwise.com/ngcp-panel-tests-selenium-stretch:latest bash
 #
 # Inside docker:
 #   su - selenium
 #   cd /code && ./t/selenium/testrunner $IP_OF_NGCP_SYSTEM # IP_OF_NGCP_SYSTEM can be e.g. IP of a `vagrant up ce-trunk` system (eth1)
 #
-# On host (where vncviewer can be e.g. from Debian package xvnc4viewer):
+# On host (where vncviewer can be e.g. from Debian package tigervnc-viewer):
 # % vncviewer geometry=1280x1024x16 localhost:5999
 ################################################################################

t/selenium/testrunner

@@ -27,18 +27,20 @@ OUTPUT_DIRECTORY="${2:-/code/}"
 echo "Setting VNC password"
 printf '%s\n%s\n\n' "${PASSWORD}" "${PASSWORD}" | vncpasswd >/dev/null
 PASSWORD_ENCODED=$(hexdump -v -e '"\\""x" 1/1 "%02X"' < "${HOME}/.vnc/passwd")
-xvnc_process=$(pgrep -f 'Xvnc4 :99' || true)
+xvnc_process=$(pgrep -f 'Xtigervnc :99' || true)
 if [ -n "${xvnc_process:-}" ] ; then
-  echo "Warning: existing VNC server found, not restarting Xvnc4 process (PID: $xvnc_process)."
+  echo "Warning: existing VNC server found, not restarting Xtigervnc process (PID: $xvnc_process)."
 else
   echo "Starting VNCserver on display :99"
-  vncserver -geometry 1280x1024 :99
+  vncserver -localhost no -geometry 1280x1024 :99
 fi
 
-# selenium
-pkill -f '/usr/bin/java -jar /home/selenium/selenium-server-standalone.jar' || true
-# NOTE: this could silently fail because selenium-server-standalone.jar is missing a daemonize option
-DISPLAY=:99 /usr/bin/java -jar /home/selenium/selenium-server-standalone.jar -trustAllSSLCertificates -log /home/selenium/selenium.log &
+echo "Firefox version:"
+firefox --version
+
+# ensure we don't leave any process behind causing problems with re-execution
+pkill -f 'geckodriver' || true
+DISPLAY=:99 ~/geckodriver &
 
 echo "################################################################################"
 echo "Finished main setup, now running tests ..."