Comparing WebDriver 2.53.1 execution for Firefox, Chrome, IE, HTMLUnit, Edge and Marionette

The Selenium 2 WebDriver with Java course has a new Continuous Integration section. Instead of using SVN we now use Git and have the source for the CI experimentation in Git (more info on this update).

As part of this process I ran all the tests on all current versions of browsers to see the status. I last performed this type of retrospective back on (WebDriver 2.42.2](http://seleniumsimplified.com/2014/06/a-quick-webdriver-2-42-2-retrospective/)

And boy have things changed with WebDriver 2.53.1.

First IE. Amazing.

I used WebDriver 2.53.1 && IEDriverServer (32 bit) v2.53.1 and only 1 of my @Test methods failed. Very impressed. It might even be an incompatibility between the HTML I use in the test page with IE rather than a driver issue, I still have to check that. Most impressed indeed.

FirefoxDriver and ChromeDriver were at parity – all @Test methods passed on both.

HTMLUnit 2.21 surprised me, and

With WebDriver 2.53.1 on Windows 10

  • FirefoxDriver && Firefox 47.0.1 – all tests passed
  • ChromeDriver v2.22.397933 && Chrome Version 52.0.2743.82 m – all tests passed
  • IEDriverServer (32 bit) v2.53.1 && IE 11.494.10586- 1 test failed
  • HTMLUnit 2.21 – 30 tests failed
  • Marionette v 0.9.0 (64 bit) && Firefox 47.0.1 – 66 tests failed
  • Edge v 10586- 71 tests failed

The HTMLUnit results could have been slightly better. HTMLUnit has a bug in version 2.21 where it can’t getText() from paragraphs where the ‘text’ is a ‘sibling’ to an element e.g.

<p id="p2" name="pName2" >
    <a name="pName2" id="a2">
    </a>
    This is b paragraph text
</p>

Without the embedded anchor tag – HTMLUnit can getText perfectly but my test pages and code are often designed to hit ‘edge’ cases or ‘hard to automate’ code.

HTMLUnit 2.22 does not have this problem. But HTMLUnit 2.22 failed its last release build on travis-ci.

I know the problem is fixed in 2.22 because I downloaded the code to see if I could fix the error and on creating a unit test to trigger the problem, the problem did not occur. So I look forward to a release of HTMLUnit 2.22 in the future. But still a good showing from HTMLUnit, much better than I expected.

Edge and Marionette are still works in progress and some of the issues are simply ‘we haven’t implemented this yet’. Edge seems very fast in operation.

So all told, a very impressive showing, and an “A+” all round I think.

This entry was posted in WebDriver. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *