controller_billing.t:
- Billing profile now gets deleted at the end.
- Check if billing profile is deleted.
Change-Id: Ia4dcbb8498f4482fe9b76568308314ceecb95d99
controller_billing.t:
- Add a $billingname at the beginning of the test. This will generate the
name "testingprofile + 5 random numbers". Will be used to properly check
if a peering group has been created/deleted.
- Rewrite create, check and edited other code to use that new variable.
Change-Id: Id9403a356b3ceb3c67e51cc3735b75ba90511a7a
controller_peering.t:
- Add a $groupname at the beginning of the test. This will generate the
name "testinggroup + 5 random numbers". Will be used to properly check
if a peering group has been created/deleted.
- Rewrite create, check and delete code to use that new variable.
Change-Id: I16489290be9780de2f457588767344080cf71b1e
controller_customer.t:
- change move_actions to move mouse over invisible buttons instead of
some element in column.
- used cleaner xpath.
Change-Id: I69b56f5e024b8392310b28490c37654e3445e620
controller_reseller.t:
- change move_actions to move mouse over invisible buttons instead of
some element in column.
- used cleaner xpath.
Change-Id: Ibb7adda18eda2dd0707e01b15c10615ccb71c8fd
controller_customer.t:
- check for PBX enviroment variable. If set to 1, it will run additional
tests for PBX enabled systems. If nothing or 0, it will run without these.
Change-Id: I95236f051b212d63cf6075cdaebd05319279a548
controller_reseller.t:
- Is now creating a new Invoice Template.
- Will check if Invoice Template has been created.
Change-Id: I0e49bfdb1d3ec5da9818a90aa397d993147bceac
controller_customer.t:
- Edited code blocks around clicking invisible buttons. Should work
more reliable now.
- replaced find_element with find_element_by_css when checking for
empty tables. Should prevent crashing.
Change-Id: Ifa7e8f85a1eec17e530032660781fa5840685edd
Common.pm:
- Added a new argument $cancel to delete_reseller() and
delete_reseller_contracts(), so it clicks on cancel instead of ok.
- Added a second move_action(), its needed because if mouse pointer is at
the exact same location as before, the buttons wont show up.
controller_reseller.t:
- implement checks to see if reseller and contract are still there if you
press cancel.
Change-Id: Id6a097314cc2fc025dac71c085c6adef98414890
controller_reseller.t:
- Modified stuff so instead of taking the fist 'active' entry,
it now uses our new reseller.
Change-Id: I0d10158ff025e6fcab791acc445aca822208f675
Common.pm:
- delete_reseller() will delete our newly created reseller.
- delete_reseller_contract() will do the same for the contract.
controller_reseller.t:
- added both new functions.
- check if reseller is actually deleted.
- check if contract is actually deleted.
Change-Id: Ic24ca53b76264f5e91e8030f47930abcc46dc11c
Common.pm:
- add create_reseller_contract(). This will create a contract for use in
create_reseller().
- add create_reseller() to really test creating a reseller.
controller_reseller.t:
- included new functions.
Change-Id: I78bcddba3dd7f4aa6949ac88ae3cc68296194b45
controller_reseller.t:
- Test will now 'create' a reseller first, then it will check the search.
(It doesnt create anything right now...)
- Searching and verifying garbage text improved.
Change-Id: Id3b9a3a12cb62df9ed5163aee86fa6a97995977e
controller_billing.t:
- Made garbage search call much simpler.
- Replace find_element with find_element_by_css, prevents crashing if
garbage text search didnt work.
- Added new ok() tests for find_element_by_css.
- Removed commented lines
Change-Id: I42cf670c47cdc22db9ba1c8079fac3e3bb5147c8
controller_domain.t:
- All ok() calls should now have correct text to print.
- Minor formatting changes.
Change-Id: I0f74039ef32b488f19b29be726102150338bb775
controller_customer.t:
- Add better check for empty table. Should prevent stale_element_exception
- Test now adds and deletes its own domain. Enables proper search testing
Change-Id: I6eed8ce85d2b118c825cd73ff46c70a3f737a587
Common.pm:
- add function delete_domain() for clean up after test run
controller_domain.t:
- implement delete_domain()
Change-Id: Ice5e902af9819ffeb683cfe6a3f86fe523d55e87
Some selenium tests depend on each other.
Example: controller_customer.t doesnt work, if controller_domain.t isnt
executed beforehand.
Common.pm is used so that selenium tests are more independent. Right now,
it contains code to create a domain.
Very useful for multithreading later on.
Change-Id: If01c7cd4d17ef66771bc64516088d3f57b02bf78
controller_customer.t:
- removing a probably unnecessary sleep call that would trigger if either
chrome or phantomjs are used.
Change-Id: I2758c2756eb9f29fcbdcfacbce9b02faef87e056
Test was crashing, because of invisible button madness.
Now it will move the cursor to the column it wants to delete.
Button shows up and is clickable now.
Change-Id: I59aa27d7ef2fe06a6caa28620341363ff99f1042
Added wait_for_text() to wait until a text appears.
Can also be used to check if text is here.
It will return true or false to be used in further testing
Change-Id: Ib28636b7757a5312652da86d3b43b9fd8547522b
XML files generated by prove will contain the time we spend in every
test case, it allows us profile our tests and fix the most slop parts.
Change-Id: I965af2d95f57d8975613da0f46ce8abf1be4c114
Jenkins uses wrapper jenkins_docker_run which will mask exit code
anyway and this is what we want currently:
we need always zero exit code here on Jenkins.
Jenkins will detect errors in produced artifacts (XML files) if any
as a result the Jenkins job will be "yellow" instead of "red".
Change-Id: I249fcde9103ee25e6798fcb835ee4a408f8f0f57
We have it already inside t/selenium/testrunner, let's remove it here
to prevent duplicate and simplify the script.
Change-Id: I184642ce18c04cd30ea1c4a8a94a4ee1f8fdac99
testrunner:
- removed -m from local Perl launch options. this would cause diag messages
to only print when the script is finished. Without it we can see in
realtime where scripts get stuck. still used for Jenkins
- can now specify which test cases should be executed in launch options.
- code cleanups
jenkins_docker_run:
- remove hardcoded old parameters as Jenkins job path all the necessary
parameters already in the proper order, we just need to proxy them into
the shell script 'testrunner' to run it under the user 'selenium'.
Change-Id: Icaced3c3ce65cd23f3037517296e49afce4a9414
controller_customer.t:
- cleaned up some ok statements to see what acutally is going on.
- Added check for adding subscribers to a customer
Change-Id: I49a8f8e671472a7875ccc6d84d1fae50691a62ae
Previously, controller_domain.t would skip the test because
there were obviously no domains after compile.
New version adds a domain and checks if the search works.
Change-Id: I8e8bad9725d253a7c578b50beded5807b5d452b1
It should help to catch the random fail in the test:
> # Go to Customers page
> # Create a Customer
> Can't use string ("no such element") as a HASH ref while "strict refs" in use at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 684.
> at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 655.
> # Tests were run but no plan was declared and done_testing() was not seen.
> # Looks like your test exited with 255 just after 5.
> Dubious, test returned 255 (wstat 65280, 0xff00)
Change-Id: I3094500b75590a284d5f28c987cadf21eb95033d
The full issue description you can find in commit message
a9399b1efd.
In short, Selenium can't press on "Edit" button because
it is being hidden on every AJAX request to backend.
(Un)fortunately the previous fix opened new ngcp-panel JS
issue reported as TT#50005: JS performs several AJAX requests
to backend and the second request still cause the same problem
as previous caused the first one.
Luckily JS performs only one request to backend on the page load.
And starts doubling them when we enter something into "filter".
As a result, the previous commit has been reverted and new approach
is proposed here: we will just wait data appearance in "Dates"
(which means the first AJAX request is finished).
Change-Id: If991af70e1ad1b00b04e4118c0b5f52b2d636eba
The previous variable name didn't represent the real code context.
The changes were initially part of commit a9399b1efd and
were extracted to separate commit as initial commit had been
reverted in commit 0601b821006f.
Change-Id: Idf06a06b60498ec5df7a9472aa38413997dc23cb
The fix here didn't work well because another ngcp-panel issue has been
noticed and reported as TT#50005, the second AJAX request aborts Selenium
test by hiding "Edit" button. The better approach is follow.
This reverts commit a9399b1efd.
Change-Id: Icf6251261f0cbc12ef4e819b3352dc3ebf7c4246
Jenkins reports time-2-time issue on 'Peering Servers' page:
> # Delete the previously created Peering Rule
> ok 14
> Can't use string ("no such element") as a HASH ref while "strict refs" in use at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 684.
> at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 655.
> # Tests were run but no plan was declared and done_testing() was not seen.
> # Looks like your test exited with 255 just after 14.
It was caused by slow AJAX loading here.
The button 'Delete' was not yet loaded and
we were just do 'sleep 1' and were trying to press it.
How we are clearly waiting for the record to be loaded
before trying to delete it.
Change-Id: Ia20cb427262e9c973ce4853d3ebb17438780afd0
It was old floating issue which randomly failed the test and
jenkins job ngcp-panel-test-selenium-docker became often yellow.
The problem here was in two facts:
- 'Edit' button available with mouseover only
- there is AJAX on this page for filtering 'Dates' records.
Historically we have no ability to wait for all AJAX loadings.
When Selenium moved mouse over Edit column to press it,
AJAX can be finished and removed "Processing..." label
from "Dates" field. It cause loosing mouseover from "Edit"
button and button is hidden causing "element not interactable":
> # Click 'Edit' button for element 'Wednesday'
> Can't use string ("element not interactable") as a HASH ref while "strict refs" in use at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 684.
> at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 655.
> # Tests were run but no plan was declared and done_testing() was not seen.
> # Looks like your test exited with 255 just after 27.
It has been workarounded here as we cannot wait for AJAX.
We are creating fake query for "Dates" (searching "thisshouldnotexist")
and waiting until the table "Dates" has no search results records.
It means AJAX has finished and we can continue with the test.
Also here I have renamed confusing button from 'edit' to 'delete',
as we press delete. And we have to clean previous search 'thisshouldnotexist'
with proper '2008-02-28' which we want to delete.
Change-Id: I55af32991687e82c3e57b15c066e772ff0b77123
It will give Ajax enough time to load 'Edit' button,
otherwise we have some mystery which randomly happens
when Jenkins tests VMs on Google Cloud:
> # Edit Wednesday
> ok 26
> # Move mouse over 'Weekdays' row to make 'Edit' button available
> # Find 'Edit' button for element 'Wednesday'
> ok 27
> # Click 'Edit' button for element 'Wednesday'
> Can't use string ("element not interactable") as a HASH ref while "strict refs" in use at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 684.
> at /home/selenium/Selenium-Remote-Driver/lib/Selenium/Remote/Driver.pm line 655.
> # Tests were run but no plan was declared and done_testing() was not seen.
> # Looks like your test exited with 255 just after 27.
Locally it is not reproducible, also VM on Proxmox has
no such issues as well. I spend too much time here trying
to find the root of the issue, maybe Firefox version
we are currently using has some issues here. Give up, sleep 1.
Change-Id: I168270f6f6d25d8bdccfe6a40b2bc63aa0c5068e
The old code didn't work as expected, as RC receives
'tee' exit code (which is zero in most of the cases)
instead of 'prove' exit code.
The new code gets 'prove' exit code and it changes nothing
at the end, as we just print it into debug message.
Anyway Jenkins operates with JUnit output to detect issues.
It is better to have 'correct' code here to
prevent misunderstanding in the future.
Change-Id: I09f399813179590a67c25509f310c91f7e297f3a
The 'Edit' button appears only after mouse-over event,
and we might have race conditions here when we are trying
to click on not-yet available/accessible button causing:
> Can't use string ("element not interactable")...
Let's wait for button and then proceed with 'click()',
it is the behaviour we use everywhere, while it was missing here.
Also improve debug messages here for easier debug next time.
Change-Id: Iae955bb3f937da6b5c10369bc49f6dd75611dc87
We can replace the apply usage with map plus a returning s///.
This also fixes a hidden bug with the liblist-moreutils-perl dependency
missing from debian/control, being indirectly pulled by some other
dependency, while an implementation detail.
Change-Id: Ibfc7b362ad77c4b202d93cf71774c40623503f14
adapt PERL5LIB in a way, so that jenkins is also able to find it
(it uses /home/selenium/jenkins_docker_run)
Change-Id: I2d5ab35c77abd2dda9f676768da8cd6a36c4ddfb
Selenium::Remote::Driver v1.20 is the latest upstream release and
is missing some patches/PRs that we need for selenium tests of
ngcp-panel. Packaging the git version properly in the Debian way
is hard because of the way upsteam handles releases with separate
branches (one of them even being out-of-date, sigh). So instead
let's use a tarball from their latest git version until we see a
v.1.21 release.
Symlinking the geckodriver binary to /usr/bin/geckodriver addresses:
| Unable to find the geckodriver binary in your $PATH. at /home/selenium/Selenium-Remote-Driver/lib/Selenium/CanStartBinary/FindBinary.pm line 66.
Change-Id: I8ddf3c76a008ce5b0aa67b62154a9614f90cb659
all selenium tests are working with newest geckodriver (0.18.0) and patched S:R:D
issues:
- setting the window size: not urgent
- the unsupported move_to: fixed (new move_action)
- setting the implicit wait timeout: fixed upstream in S:R:D
Change-Id: Ic5b52cd866f4fd93682d67a4fd8a76ec54ea70cf
This package update includes further patches to address Selenium
test failures with geckodriver/Firefox:
* move_to: 4d0b8e7255
* timeouts: 2c55f3dcb9
Change-Id: I587f32230ce7cc8d912e052a93fb7d31fd7c90c1
Which confuses developers which are debugging failed tests,
as details like below are not a part of XML Jenkins displays on a WEB:
> # Failed test 'create test customer w/o profile package'
> # at t/api-rest/api-balanceintervals.t line 1578.
> # got: '500'
> # expected: '201'
> Use of uninitialized value in concatenation (.) or string at t/api-rest/api-balanceintervals.t line 1580.
> Thread 4 terminated abnormally: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "<!DOCTYPE HTML>\n<!-...") at /usr/share/perl5/JSON.pm line 171.
'man prove' says:
> -m, --merge Merge test scripts' STDERR with their STDOUT.
Change-Id: I945d4d8867fc9c061f96d067be14125af7cc5a46
An unintended docker image rebuild of
ngcp-panel-tests-selenium-jessie resulted in failures with
our selenium tests. A series of problems started:
1) Updating the Selenium version resulted in an error message by
Selenium, requesting the geckodriver binary
(https://github.com/mozilla/geckodriver - being a proxy for using
W3C WebDriver-compatible clients to interact with Gecko-based
browsers). So Selenium versions >=3 depend on geckodriver.
2) Recent versions of Selenium also depend on Java >=8, available
only via backports for jessie. Since we avoid usage of backports
as much as possible and we'd have to upgrade once anyway we
decided to base our fresh docker image on stretch.
3) We have to backport Selenium::Remote::Driver (Debian package
libselenium-remote-driver-perl) for stretch (since it's not
available in Debian, even though there's an ITP:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839569). It has
a bug which prevents its usage for us:
https://github.com/gempesaw/Selenium-Remote-Driver/issues/319 ->
We fixed the problem via our own backport and submitted a PR
towards upstream:
https://github.com/gempesaw/Selenium-Remote-Driver/pull/321
4) Firefox from Debian/stretch (firefox-esr 52.2.0esr-1~deb9u1)
fails with geckodriver e.g. in click events, the current Firefox
version 54 works fine though. So instead install only the
relevant runtime packages from Debian and use the Firefox version
v54 from upstream.
Resulting changes:
* switch from Debian/jessie to Debian/stretch
* no longer use selenium at all but only geckodriver instead
(recent versions of selenium depend on geckodriver anyway)
* do all package/file changes in /home/selenium/ (to clearly
see what's from us/docker and what are actual temporary files)
* testrunner related changes:
- invoke vncserver with `-localhost no` option to accept
VNC connections not only from localhost only
- adjust vncserver process name for Xvnc4->Xtigervnc switch
- start geckodriver instead of selenium
* package related changes:
- backported libtest-pretty-perl (v0.32)
- backported libselenium-remote-driver-perl (v1.20)
- add curl to package list since it's handy for debugging test
failures, like e.g. `curl -X POST localhost:4444/session --data {}`
- add procps to have pkill binary available inside testrunner
- openjdk/java is no longer needed since we don't depend on selenium anymore
- vnc4server is now a transitional package for
tigervnc-standalone-server, so directly depend on it
Change-Id: I607ffd4fce80f9e84b76c3c7a19824819ac4503c
File /.dockerinit no longer exists. With Docker versions 1.11 and
1.12 there's /.dockerenv available, though it's not yet sure
whether this will be present in the future too, so check for either
/.dockerenv or a proper cgroup setup inside the docker container.
File /proc/1/cgroup with Docker version 1.12.1 on Debian/jessie
looks like:
| root@4c3a1c6d773c:/# cat /proc/1/cgroup
| 8:freezer:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 7:perf_event:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 6:net_cls,net_prio:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 5:cpuset:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 4:blkio:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 3:cpu,cpuacct:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 2:devices:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
| 1:name=systemd:/docker/4c3a1c6d773c1a5675f24ff58540ff1a290489e7325e1e66ec2c3392aa3a1145
See https://github.com/docker/docker/issues/18355 and
https://github.com/docker/docker/pull/19490 for details.
Change-Id: I79fff0eead1a11f80b461791b2285a65d706262a
The jenkins_docker_run script is invoked from Jenkins
when running the docker image. The testrunner script
now takes two additional arguments to support sharing
test results via another docker volume and to control
output format ("pretty" as default but junit for usage
on Jenkins).
While at it fix some quoting issue identified by shellcheck.
Change-Id: Ided716e0e0ca23751b0ee2961f3e301acb80128d
This provides a setup to run ngcp-panel tests against
selenium-server using docker. It's based on Debian/jessie and
assumes are reachable NGCP system for running the tests against
(e.g. via `vagrant up ce-trunk`). For usage instructions refer
to the Dockerfile.
NOTE: replacing 'jessie' with 'wheezy' in the Dockerfile provides
a setup based on Debian/wheezy.
Change-Id: I79ca4d44c3a7c454f99148889d4b0d075a79e82b
use strict and warnings
therefore forbid unquoted strings
factor out initialisation
screenshot functionality is implemented upstream
Change-Id: I18a0dcc7a6418dc9fd2fa64d367ae226018a7645
Nico Schedel
Alexander Lutay
Irina Peshinskaya
Guillem Jover
Gerhard Jungwirth
Michael Prokop
Victor Seva
Andreas Granig