First impressions of Selenium WebDriver 3.0.0 beta 1

TLDR; Selenium 3 WebDriver is pretty much a drop in replacement for Selenium 2 WebDriver

The 3.0.0 Beta v1 version of WebDriver has been released and my first Impressions of Selenium 3 beta are very good.

And first impressions means, can I run the stuff I have already.

I amended the pom.xml to bring the release down from maven central.

<!-- <webdriver.version>2.53.1</webdriver.version> -->
<webdriver.version>3.0.0-beta1</webdriver.version>

So I effectively included it as

<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-server</artifactId>
    <version>3.0.0-beta1</version>
</dependency>

Since WebDriver now needs Java 1.8 so I’ll bump the language dependencies in pom.xml

<configuration>
   source>1.8</source>
   <target>1.8</target>
</configuration>

In my test code I’ve been using a base64 encoding implementation bundled with Selenium, for working with screenshots

//import org.bouncycastle.util.encoders.Base64;

Since WebDriver now needs Java 1.8, I’ll just use the decoder in Java 1.8 instead

//Base64 decoder = new Base64();
//byte[] imgBytes = (byte[]) decoder.decode(tempImageFileAsBase64);
byte[] imgBytes = Base64.getDecoder().decode(tempImageFileAsBase64);

With WebDriver 3.0.0-beta1 on Windows 10

  • FirefoxDriver && Firefox 47.0.1 – all tests passed
  • ChromeDriver v2.22.397933 && Chrome Version 52.0.2743.82 m – all tests passed
  • IEDriverServer (32 bit) v2.53.1 && IE 11.494.10586- 1 test failed
  • HTMLUnit 2.21 – 28 tests failed
  • Marionette v 0.9.0 (64 bit)&& Firefox 47.0.1 – 65 tests failed
  • Edge v 10586- 70 tests failed

3.0.0-beta1 defaults to Marionette, so to use the internal FirefoxDriver we can

  • set a capability
DesiredCapabilities legacyCapabilities = DesiredCapabilities.firefox();
legacyCapabilities.setCapability("marionette", false);
WebDriver aDriver = new FirefoxDriver(legacyCapabilities);

Code suggests that the plan is to allow us to:

  • set a system property “webdriver.firefox.marionette”

But I couldn’t get that to work:

  • -Dwebdriver.firefox.marionette=false
  • System.setProperty(FirefoxDriver.SystemProperty.DRIVER_USE_MARIONETTE, "false");

It doesn’t look like that has been coded for yet. Although I can now see that this was present in 2.53.1 as well (I didn’t notice that before)(I couldn’t get it to work in 2.53.1 either).

So – set the capability for now to switch between Marionette and legacy driver in Selenium WebDriver 3.0.0-beta1.

The MarionetteDriver that I’ve been using will be deprecated and we’ll use the FirefoxDriver for both Marionette and Legacy – with just a toggle of the capability.

And with that I guess I’m ready to migrate to 3.0.0 when it moves out of beta.

Pretty much the drop in replacement that the selenium team have been working towards. Another A+

Posted in WebDriver | Leave a comment

Comparing WebDriver 2.53.1 execution for Firefox, Chrome, IE, HTMLUnit, Edge and Marionette

The Selenium 2 WebDriver with Java course has a new Continuous Integration section. Instead of using SVN we now use Git and have the source for the CI experimentation in Git (more info on this update).

As part of this process I ran all the tests on all current versions of browsers to see the status. I last performed this type of retrospective back on (WebDriver 2.42.2](http://seleniumsimplified.com/2014/06/a-quick-webdriver-2-42-2-retrospective/)

And boy have things changed with WebDriver 2.53.1.

First IE. Amazing.

I used WebDriver 2.53.1 && IEDriverServer (32 bit) v2.53.1 and only 1 of my @Test methods failed. Very impressed. It might even be an incompatibility between the HTML I use in the test page with IE rather than a driver issue, I still have to check that. Most impressed indeed.

FirefoxDriver and ChromeDriver were at parity – all @Test methods passed on both.

HTMLUnit 2.21 surprised me, and

With WebDriver 2.53.1 on Windows 10

  • FirefoxDriver && Firefox 47.0.1 – all tests passed
  • ChromeDriver v2.22.397933 && Chrome Version 52.0.2743.82 m – all tests passed
  • IEDriverServer (32 bit) v2.53.1 && IE 11.494.10586- 1 test failed
  • HTMLUnit 2.21 – 30 tests failed
  • Marionette v 0.9.0 (64 bit) && Firefox 47.0.1 – 66 tests failed
  • Edge v 10586- 71 tests failed

The HTMLUnit results could have been slightly better. HTMLUnit has a bug in version 2.21 where it can’t getText() from paragraphs where the ‘text’ is a ‘sibling’ to an element e.g.

<p id="p2" name="pName2" >
    <a name="pName2" id="a2">
    </a>
    This is b paragraph text
</p>

Without the embedded anchor tag – HTMLUnit can getText perfectly but my test pages and code are often designed to hit ‘edge’ cases or ‘hard to automate’ code.

HTMLUnit 2.22 does not have this problem. But HTMLUnit 2.22 failed its last release build on travis-ci.

I know the problem is fixed in 2.22 because I downloaded the code to see if I could fix the error and on creating a unit test to trigger the problem, the problem did not occur. So I look forward to a release of HTMLUnit 2.22 in the future. But still a good showing from HTMLUnit, much better than I expected.

Edge and Marionette are still works in progress and some of the issues are simply ‘we haven’t implemented this yet’. Edge seems very fast in operation.

So all told, a very impressive showing, and an “A+” all round I think.

Posted in WebDriver | Leave a comment

Using Intellij to debug @Test with the ‘Watch’ function

I was sent a query about why a cookies test was failing on Chrome but not on Firefox.

The debug process involved me using techniques that I don’t think I have blogged about or demonstrated before so I repeated the debug session for a video and have written it up here.

Essence:

  • Run test on Firefox to make sure it does work
  • Change Run configuration in IntelliJ to configure Chrome as the browser
  • Breakpoints on relevant lines
  • Add a watch that uses ‘code’ not just a variable name
  • Step through the code comparing my expectations with the actual values shown in the variables and watch view

I have a `Driver` class, which allows me to configure which driver I use by a method call, or environment variables or properties.

The fact that it uses properties means that if I have a run configuration in IntelliJ I can add a `-D` parameter to the VM options to configure my browser

  • `-Dselenium2basics.webdriver=CHROME`

The ‘watch’ functionality in the debug view can be used to watch variables, but it can also be used to ‘execute arbitrary code and watch the results’ after each ‘step’ in the debug process.

I setup a ‘watch’ on `driver.manage().getCookies()` to see the cookies retrieved and set by Selenium WebDriver on each step.

By comparing the cookies that I created, with the cookies that were added, I could see that the cookie created by `driver.manage().addCookie` through the ChromeDriver added a `.` before the domain in the cookie.

i.e. I created a cookie with domain `compendiumdev.co.uk` it was added to the brower as `.compendiumdev.co.uk`

This led to the side-effect that:

  • the JavaScript on the page could access the cookie fine
  • WebDriver could get the cookie just fine
  • when the JavaScript on the page amended the cookie, it created a new cookie

I have to perform a follow on action to see:

  • if I need to fix the code on my test page, since the JavaScript might not be ‘amending the cookie properly’ or it might have a bug deleting the cookie, etc. or
  • if it is a bug in ChromeDriver (different effect between FirefoxDriver and ChromeDriver suggests it might be a bug)

I don’t know yet. But the debug process identified the cause of the side-effect, now I have to work on the fix.

You can see the debug session in the video below, and the two blog posts listed below might be useful if you are debugging Selenium WebDriver and Java @Test code:


P.S. I decided to absorb the ‘fix’into my test pages, so that the cookie deletion code doesn’t just expire it for the current domain, it also expires it for  `”.” + document.domain`

 document.cookie = name + "=" + cval + "; expires=" + exp.toGMTString();
 document.cookie = name + "=" + cval + "; domain=." + document.domain + "; expires=" + exp.toGMTString();
Posted in Java, WebDriver | Leave a comment

Appium – a very short getting started guide

I recently amended the Selenium WebDriver API with Java online training to include a section on Appium with Android.

In essence, Appium for mobile web testing is pretty simple, you simply use RemoteWebDriver to connect to the Appium server.

Use the capabilities to setup the remote webdriver.

DesiredCapabilities capabilities = new DesiredCapabilities();

capabilities.setCapability("deviceName", "emulator-5554");
capabilities.setCapability("platformName", Platform.ANDROID);
// if you are using an up to date version of android to you can use "Chrome" as the app,
// otherwise "browser" is the default 'browser' on Android
capabilities.setCapability("app", "browser");

String appiumURL = "http://127.0.0.1:4723/wd/hub";
driver = new RemoteWebDriver(
        new URL(appiumURL),
        capabilities);

// and then use driver as normal

To get started with Appium:

If you are working with modern physical device connected via USB then you can probably use "Chrome" as your "app" and then drive the Mobile Chrome browser (which handles frames better than the default ‘browser’.

If you are working with the Android VM and an AVD then you’ll probably have to use Level 23 of the API, which will come with ‘browser’ rather than ‘chrome’.

Android N (api level 24) comes with Chrome in the AVD but you’ll have to amend Appium slightly to work with it (I chose not to do that and used level 23 with the AVD, and used Chrome on the physical devices).

We cover all of this in more detail on the course but each of the sites linked to above, has install and getting started instructions.

And if you get everything setup, just connect as a RemoteWebDriver as shown in the code snippet above.

You might have to amend the @Test code a little to introduce different synchronisation, but I was surprised at how compatible it was.

Appium has come a long way in a short time, has overcome enormous challenges and deserves to be the default mobile bridge for Selenium WebDriver.

Posted in WebDriver | Leave a comment

How to investigate and debug a Selenium WebDriver `@Test` error

Here’s the situation. You’ve written some code to automatically execute your application. You’re using Selenium WebDriver.

Bam.

Something goes wrong.

You don’t know what to do.

Do you:

  • A) investigate?
  • B) ask someone else to fix your problem?

If you answered B to the question above, then in this post I’ll show you some strategies to help with (A).

You do have to develop the skills to investigate your own issues. Learning how to investigate will help you build those skill sets so you can do your job better. It doesn’t mean that you never ask for help, but it means you can have more self-reliance and when you ask for help you will have more information to provide the person you have asked for help.

Bing the error to Google

Don’t make the problem something that the other person can google for you

Use your favourite web search engine to search for the error message.

Hints:

  • Copy and paste the exact text
    • do not write it using your own words based on what you think it means
    • because you don’t know what it means, that’s why your searching
  • You may have to copy and paste part, or parts, of the error message
    • your error may have appeared on a specific line number, don’t include that in the search
      your error message may be related to specific environment variable, don’t include that in the search
  • put quotes " around the text that is important

Building a search for the error message is a skill. But you will get better at it over time if you try.

  • Read multiple pages and answers
    • do not assume that the first answer you encounter is the correct one
  • put in the effort to understand the answer
    • it is possible that you might not understand the answer, that means you might have to do extra web searches based on the answer. Do not stop at this point and jump to (B). Do the research to help you understand the answer, otherwise you will never learn how to do this for yourself.

Choosing the correct answer is a skill. But you will get better at it over time if you try. reference

Experiment with plausible answers

‘their’ answer may not work for you.

You will need to experiment and possibly amend the code that you saw in the answer.

This is a skill, you need to develop over time. reference

Debug and Step Through the code

The above links have additional information and videos on how to use debuggers.

  • Set a breakpoint at the line that throws the error or exception.

Run the test in debug mode.

When the code stops executing at the breakpoint. Check that the application is in the correct state i.e. all elements are ready, you can see the element you want to work with etc.

Then step over the line that throws the execption.

If it passes then chances are you need some synchronisation code immediately prior to your statement to wait for the element to be ready. Because that is what you did when you debugged: you waited, then you executed the code. You want to add the wait into your code.

And make it an application state wait i.e. you checked to see if the element was present, was visible, was in the state you needed. Wait for that in the code.

If that didn’t work then:

  • Set a breakpoint a few lines before the line that throws the error or exception.

Repeat your debug process, carefully watching what is happening. Making sure that the application state is being setup the way that you require it for your code to work, i.e. you click on previous controls, something appears, the data is entered etc.

You are making sure that the previous lines in the test code actually setup the pre-conditions that you need in place for the line of code that is currently throwing an exception.

You might have written something incorrectly prior to the line.

You might need to have more synchronisation, prior to the line.

Are you sure it is working as you expect?

As you debug the code and step through the lines, make sure:

  • the variables have the values you expect
  • none of the variables are null
  • it selected the correct element
  • etc.

You can do this by watching in the variables tab, using the EvaluateExpression functionality in the IDE, System.out.println variables and values to the console etc.

Are you sure that locator is correct?

Manually use the application and copy paste the locators from your code into whatever tool you are using on your computer to check locators:

  • Firebug
  • Developer tools search
  • document.evaluate
  • document.querySelector or document.querySelectorAll

Make sure you copy and paste from the code to get ‘exactly’ the same in the browser as you are using in the code. You might be amazed at how often you can fix typos when transcribing from one source to another (but only when debugging, the reverse is true (inject typos) when adding to your code).

Essentially – find another way of using the code to check if it works.

If you do have to resort to (B)

If you do have to ask someone then make sure:

  • You have tried all of the above so you can tell them what you tried.

That way:

  1. They know you tried
  2. They know what didn’t work
  • Make sure your example is the smallest code example that recreates the problem.

That way the problem is not related to some other part of the code.

By the way – doing this can often help you find the problem.

  • Make your error report detailed

e.g.

  • what line of code throws the error
  • what versions of language, webdriver, browser?
  • intermittent? always?
  • just this app, or every app

By thinking through the environment and situation you might also come up with other ideas of what to try to continue investigating.

If you resort to (B) on the internet

Here are some tips from:

Posted in Java, WebDriver | Leave a comment

Use Selenium WebDriver 2.53.1 with Firefox 47.0.1

TL;DR Upgrade to Firefox 47.0.1 and use Selenium WebDriver 2.53.1 (locally at the moment) to use the FirefoxDriver

I suspect this will be a fairly short lived post since Selenium WebDriver 2.53.1 will be in maven central soon. But longer term this post will act as an example of how to use a version of Selenium WebDriver that you have downloaded, without it being present in maven central repositories.

As per this tweet by Jim Evans:

Hooray. Selenium WebDriver 2.53.1 is once more compatible with `FirefoxDriver`.

So:

Since this is a temporary hack, until 2.53.1 becomes available in maven, I just added the jar file to my downloads folder and amended the pom.xml to use, as a local dependency, the downloaded version:

<!--
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>2.53.0</version>
</dependency>
-->

<dependency>
    <groupId>selenium_2_53_1</groupId>
    <artifactId>com.seleniumhq.selenium_2_53_1 </artifactId>
    <version>2.53.1</version>
    <scope>system</scope>
    <systemPath>
     C:/Users/Alan/Downloads/selenium-2.53.1/selenium-server-standalone-2.53.1.jar
    </systemPath>
</dependency>

Essentially I created a dependency on a local file in the system, rather than from the maven repo. There are a bunch of ways of making 2.53.1 available to maven locally, but this is the approach I tend to fall back into, because I’m usually using it for quick hacks.

When Selenium 2.53.1 is available in maven I won’t need the systemPath dependency. I’ll just un-comment back the Selenium 2.53.0 dependency, amend it to 2.53.1, and remove the `systemPath` dependency.

You could clearly add this into your resources or wherever else you want to store it. You could also add it into a local maven repo – see related links below.

I also updated from firefox 47.0 to 47.0.1

Note: You need to update both Firefox to 47.0.1 and Selenium WebDriver to 2.53.1

Then your test code should run as normal, simply by instantiating FirefoxDriver

When Selenium WebDriver 2.53.1 is available in maven then you won’t need the local pom.xml hack.

Thanks to the Selenium and Mozilla team for all their work involved in fixing this.

Remember you can also: drop down to Firefox 45 Extended Support Edition, Use portable Firefox, Use the Marionette/GeckoDriver, see the following blog posts for more details:

Related Links:

Options for using local jar files in a maven project

Posted in WebDriver | Leave a comment

Using Portable Firefox 46 with WebDriver

As mentioned on David Burns’ blog and in a previous post here Firefox are moving towards Marionette driver. At the time of writing Firefox 47 doesn’t work with FirefoxDriver and you need to use Marionette. This may change back soon, and David’s blog post might no longer be available.

As mentioned on the above posts you can use Marionette, or use the ESR.

OR you could use a portable version of Firefox.

Rather than try and toggle multiple versions of Firefox. I downloaded a portable version of Firefox. I used a version distributed by portableapps.com. Having a portable app means I can run 46 and 47 at the same time and don’t have to uninstall 47 to use 46. The portable app puts all its DLLs in the app folder and doesn’t really ‘install’ it mostly un-archives itself.

On the sourceforge site for portableapps they have a set of older Firefox versions:

I downloaded a version of Firefox 46.0.1

I extracted/installed this to my ‘tools’ folder which is a peer of my ‘code’ folder:

  • code
    • pom.xml
    • src
  • tools
    • chromedriver
    • marionette
    • iedriver_64
    • etc.

And now I have a FirefoxPortable folder in my tools folder.

When I instantiate a FirefoxDriver I no longer write:

WebDriver firefox = new FirefoxDriver();

Because that would trigger the “Firefox is no longer working” error.

Instead, I tell the FirefoxDriver which executable to use:

FirefoxProfile profile = new FirefoxProfile();
firefox = new FirefoxDriver(
                 new FirefoxBinary(
                     new File(System.getProperty("user.dir"),
                     "/tools/FirefoxPortable/FirefoxPortable.exe")
                  ), 
               profile);

And then I can run my @Test code as normal without having to use Marionette.

Obviously you can abstract this instantiation into a class so you don’t have to change code everywhere.

I have updated my ‘start using selenium webdriver’ github repo to have examples of the different approaches for instantiating Firefox.

 

Posted in Firefox, WebDriver | Leave a comment

How to use the Firefox Marionette Driver

Note: This post was triggered by a blog post on David Burns’ Blog which states that FirefoxDriver will stop working at version 47 of Firefox. 

It is easy to switch to using the new MarionetteDriver.

  • I suggest you try it on your code, and if your tests work, then keep using it.
  • If your tests fail, then it is most likely some defects in the newer MarionetteDriver. In which case:

Steps to switch:

  • Download the marionette driver
  • Unarchive it to a folder of your choice
  • In your code set the "webdriver.gecko.driver" property to the location of the .exe
  • Use the MarionetteDriver

The above is slightly different from the official instructions on the page.

The official instructions suggest you:

  • rename the file to ‘wires.exe’ and
  • add it to your path.

I did rename it to wires.exe, mainly to keep the code consistent so I wouldn’t have to update the code everytime I change the version of the marionette driver.

But I didn’t add it to the path. I’d rather control that in the code when the option is provided.

So for Firefox 45 and 46 my start driver code looks like this:

WebDriver driver = new FirefoxDriver();

And for Firefox 47 and above my code looks like this:

String currentDir = System.getProperty("user.dir");
String marionetteDriverLocation = currentDir + "/tools/marionette/wires.exe";
System.setProperty("webdriver.gecko.driver", marionetteDriverLocation);
WebDriver driver = new MarionetteDriver();

I have a working example of this in the startUsingSeleniumWebDriver code on github:

Posted in FAQ, WebDriver | 6 Comments

Coding for workarounds so you know when they are fixed

With most libraries you use to automate your work, you have to code workarounds.

I coded a workaround in my Selenium WebDriver tests because there was a bug in ChromeDriver which meant that cookies were created differently than other browsers.

When I upgraded to WebDriver 2.53.0 and ChromeDriver 2.21 I discovered I didn’t need my workaround anymore.

In the code I had one set of code for Chrome, and another set for other browsers:

// chromedriver has a bug where the domain is prefixed with '.' 
//therefore
// the app 'updates' a new cookie so when chromedriver
// returns the cookie it returns
// the first and not the updated one
if(Driver.currentBrowser()== Driver.BrowserName.GOOGLECHROME){
  aCookie = getCookieWithLargestValueNamed
               ("seleniumSimplifiedSearchNumVisits");
}else {
  aCookie = driver.manage().getCookieNamed(
                "seleniumSimplifiedSearchNumVisits");
}

The bug was 7499 and was fixed in the recent update to ChromeDriver.

How do I know it was fixed?

Because I had failing @Test methods.

Tests in error:
  changeCookieVisitsCountUsingCookieBuilder
  (com.seleniumsimplified.webdriver.cookies.
   CookiesExercisesTestWorkWithExtraSync): 
  Chrome cookie creation issue has been resolved
  changeCookieVisitsCount(
    com.seleniumsimplified.webdriver.cookies.
    CookiesExercisesTestWorkWithExtraSync): 
  Chrome cookie creation issue has been resolved

Why did they fail (the workaround was still valid)?

Because I had code that checked for that.

// This currently fails on Chrome Driver since
// ChromeDriver adds some extra cookies when it creates 
// a cookie with a domain
// https://code.google.com/p/selenium/issues/detail?id=7499
if(Driver.currentBrowser() != Driver.BrowserName.GOOGLECHROME){
  assertEquals("we added the cookie correctly", cookieCount, 
               driver.manage().getCookies().size());
}else{
  if(driver.manage().getCookies().size()==cookieCount){
    throw new RuntimeException(
                 "Chrome cookie creation issue has been resolved");
  }
// also Chromedriver now adds a '.' in front of the domain
}

I added extra code to throw an error, if the problem I was coding a workaround for went away.

I don’t know if that is a normal practice. But since I don’t really want workarounds in my code forever, and I likely won’t remember to check the release notes on every fix to see if I coded a workaround for a fix. This seems like a good workaround to check if workarounds are no longer good.

The code I was using to check for the largest value in a cookie, by the way, was:

private Cookie getCookieWithLargestValueNamed(String cookieName) {
    int largestCookieVal=0;
    Cookie largestCookie=null;
    for(Cookie aCookie : driver.manage().getCookies()){
        if(aCookie.getName().contentEquals(cookieName)){
            int cookieVal = Integer.parseInt(aCookie.getValue());
            if(cookieVal>largestCookieVal) {
                largestCookieVal = cookieVal;
                largestCookie = aCookie;
            }
        }
    }
    return largestCookie;
}

 

Posted in Practices, WebDriver | 5 Comments

Upgrade to Selenium WebDriver 2.53.0 and ChromeDriver 2.21

Selenium WebDriver 2.53.0 is available
Selenium WebDriver 2.53.0 is available

Upgrade to Selenium WebDriver 2.53.0

I’ve just updated my online Selenium 2 WebDriver with Java course to use version 2.53.0 of Selenium WebDriver. Change Log

I mainly update due to browser changes but this update required a code fix.

Selenium WebDriver 2.53.0

Minor code fix in the pom.xml to add a dependency on the new Htmlunit driver project.

github.com/SeleniumHQ/htmlunit-driver

The package structure in the htmlunit driver was left the same so all I had to amend was the pom file

<webdriver.version>2.53.0</webdriver.version>
<junit.version>4.11</junit.version>
<hamcrest.version>1.3</hamcrest.version>
<htmlunitdriver.version>2.20</htmlunitdriver.version>

And the include for HTMLUnit became:

<!-- from WebDriver 2.53.0 HTMLUnit is no longer distributed with Selenium WebDriver -->
<!-- https://github.com/SeleniumHQ/htmlunit-driver -->
<!-- need to bring in the htmlunit driver on its own -->
<!-- https://github.com/SeleniumHQ/selenium/blob/master/java/CHANGELOG -->
<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>htmlunit-driver</artifactId>
    <version>${htmlunitdriver.version}</version>
</dependency>

ChromeDriver 2.21

I also had to update my build machine to use 2.21 ChromeDriver.

When I did this I discovered that this version has fixed some Cookie bugs when working against Google Chrome.

code.google.com/p/selenium/issues/detail?id=7499

So I amended my tests to cater for this as well.

I’ve uploaded the code to my online training course existing students can download the amended code from the course materials section.

Posted in Training Courses, WebDriver | Leave a comment