Category Archives: Java

Using Intellij to debug @Test with the ‘Watch’ function

I was sent a query about why a cookies test was failing on Chrome but not on Firefox.

The debug process involved me using techniques that I don’t think I have blogged about or demonstrated before so I repeated the debug session for a video and have written it up here.

Essence:

  • Run test on Firefox to make sure it does work
  • Change Run configuration in IntelliJ to configure Chrome as the browser
  • Breakpoints on relevant lines
  • Add a watch that uses ‘code’ not just a variable name
  • Step through the code comparing my expectations with the actual values shown in the variables and watch view

I have a `Driver` class, which allows me to configure which driver I use by a method call, or environment variables or properties.

The fact that it uses properties means that if I have a run configuration in IntelliJ I can add a `-D` parameter to the VM options to configure my browser

  • `-Dselenium2basics.webdriver=CHROME`

The ‘watch’ functionality in the debug view can be used to watch variables, but it can also be used to ‘execute arbitrary code and watch the results’ after each ‘step’ in the debug process.

I setup a ‘watch’ on `driver.manage().getCookies()` to see the cookies retrieved and set by Selenium WebDriver on each step.

By comparing the cookies that I created, with the cookies that were added, I could see that the cookie created by `driver.manage().addCookie` through the ChromeDriver added a `.` before the domain in the cookie.

i.e. I created a cookie with domain `compendiumdev.co.uk` it was added to the brower as `.compendiumdev.co.uk`

This led to the side-effect that:

  • the JavaScript on the page could access the cookie fine
  • WebDriver could get the cookie just fine
  • when the JavaScript on the page amended the cookie, it created a new cookie

I have to perform a follow on action to see:

  • if I need to fix the code on my test page, since the JavaScript might not be ‘amending the cookie properly’ or it might have a bug deleting the cookie, etc. or
  • if it is a bug in ChromeDriver (different effect between FirefoxDriver and ChromeDriver suggests it might be a bug)

I don’t know yet. But the debug process identified the cause of the side-effect, now I have to work on the fix.

You can see the debug session in the video below, and the two blog posts listed below might be useful if you are debugging Selenium WebDriver and Java @Test code:


P.S. I decided to absorb the ‘fix’into my test pages, so that the cookie deletion code doesn’t just expire it for the current domain, it also expires it for  `”.” + document.domain`

 document.cookie = name + "=" + cval + "; expires=" + exp.toGMTString();
 document.cookie = name + "=" + cval + "; domain=." + document.domain + "; expires=" + exp.toGMTString();
Posted in Java, WebDriver | Leave a comment

How to investigate and debug a Selenium WebDriver `@Test` error

Here’s the situation. You’ve written some code to automatically execute your application. You’re using Selenium WebDriver.

Bam.

Something goes wrong.

You don’t know what to do.

Do you:

  • A) investigate?
  • B) ask someone else to fix your problem?

If you answered B to the question above, then in this post I’ll show you some strategies to help with (A).

You do have to develop the skills to investigate your own issues. Learning how to investigate will help you build those skill sets so you can do your job better. It doesn’t mean that you never ask for help, but it means you can have more self-reliance and when you ask for help you will have more information to provide the person you have asked for help.

Bing the error to Google

Don’t make the problem something that the other person can google for you

Use your favourite web search engine to search for the error message.

Hints:

  • Copy and paste the exact text
    • do not write it using your own words based on what you think it means
    • because you don’t know what it means, that’s why your searching
  • You may have to copy and paste part, or parts, of the error message
    • your error may have appeared on a specific line number, don’t include that in the search
      your error message may be related to specific environment variable, don’t include that in the search
  • put quotes " around the text that is important

Building a search for the error message is a skill. But you will get better at it over time if you try.

  • Read multiple pages and answers
    • do not assume that the first answer you encounter is the correct one
  • put in the effort to understand the answer
    • it is possible that you might not understand the answer, that means you might have to do extra web searches based on the answer. Do not stop at this point and jump to (B). Do the research to help you understand the answer, otherwise you will never learn how to do this for yourself.

Choosing the correct answer is a skill. But you will get better at it over time if you try. reference

Experiment with plausible answers

‘their’ answer may not work for you.

You will need to experiment and possibly amend the code that you saw in the answer.

This is a skill, you need to develop over time. reference

Debug and Step Through the code

The above links have additional information and videos on how to use debuggers.

  • Set a breakpoint at the line that throws the error or exception.

Run the test in debug mode.

When the code stops executing at the breakpoint. Check that the application is in the correct state i.e. all elements are ready, you can see the element you want to work with etc.

Then step over the line that throws the execption.

If it passes then chances are you need some synchronisation code immediately prior to your statement to wait for the element to be ready. Because that is what you did when you debugged: you waited, then you executed the code. You want to add the wait into your code.

And make it an application state wait i.e. you checked to see if the element was present, was visible, was in the state you needed. Wait for that in the code.

If that didn’t work then:

  • Set a breakpoint a few lines before the line that throws the error or exception.

Repeat your debug process, carefully watching what is happening. Making sure that the application state is being setup the way that you require it for your code to work, i.e. you click on previous controls, something appears, the data is entered etc.

You are making sure that the previous lines in the test code actually setup the pre-conditions that you need in place for the line of code that is currently throwing an exception.

You might have written something incorrectly prior to the line.

You might need to have more synchronisation, prior to the line.

Are you sure it is working as you expect?

As you debug the code and step through the lines, make sure:

  • the variables have the values you expect
  • none of the variables are null
  • it selected the correct element
  • etc.

You can do this by watching in the variables tab, using the EvaluateExpression functionality in the IDE, System.out.println variables and values to the console etc.

Are you sure that locator is correct?

Manually use the application and copy paste the locators from your code into whatever tool you are using on your computer to check locators:

  • Firebug
  • Developer tools search
  • document.evaluate
  • document.querySelector or document.querySelectorAll

Make sure you copy and paste from the code to get ‘exactly’ the same in the browser as you are using in the code. You might be amazed at how often you can fix typos when transcribing from one source to another (but only when debugging, the reverse is true (inject typos) when adding to your code).

Essentially – find another way of using the code to check if it works.

If you do have to resort to (B)

If you do have to ask someone then make sure:

  • You have tried all of the above so you can tell them what you tried.

That way:

  1. They know you tried
  2. They know what didn’t work
  • Make sure your example is the smallest code example that recreates the problem.

That way the problem is not related to some other part of the code.

By the way – doing this can often help you find the problem.

  • Make your error report detailed

e.g.

  • what line of code throws the error
  • what versions of language, webdriver, browser?
  • intermittent? always?
  • just this app, or every app

By thinking through the environment and situation you might also come up with other ideas of what to try to continue investigating.

If you resort to (B) on the internet

Here are some tips from:

Posted in Java, WebDriver | 2 Comments

Generic Selenium WebDriver Grid Configuration Handling to use BrowserStack, Saucelabs and TestingBot

Image taken from page 103 of ‘Der Beobachter. Allgemeine Anleitung zu Beobachtungen über Land und Leute.

In my Selenium WebDriver with Java course I have a Driver.java abstraction class.

I use an abstraction class so that in the @Test methods we make a call to Driver.get rather than individual firefox or chrome drivers, or RemoteDrivers

And the configuration of this is adjusted via the actual code, or a mix of environment variables and properties.

I was looking at some Mac Grid issues that I was helping Brian Long investigate. And I realised that my Generic Grid handling wasn’t as Generic as it needed to be to support my debugging.

I amended my code so that I could configure my Driver.java with either environment variables or properties:

  • Environment Variables are handy because they are system wide and you can configure them on the machine, but they are a bit of a pain to work with because you have to restart the console and IDE when you change them.
  • Properties are handy because you can pass them into mvn test jobs with -D parameters e.g. mvn test -Dparam=value and this makes it very easy to use them in CI

I primarily use IntelliJ and in the Run \ Edit Configuration section you can override environment variables, and pass in -D params so it is possible to work with these in the IDE.

I find it easier to override environment variables in the IDE because the GUI makes it easy to amend them in a list.

I find it easier to configure the CI jobs through properties by passing them in as -D to mvn test

Therefore I amended my Driver.java code to not care if its configuration comes from environment variables or properties

e.g.

String gridBrowser = getPropertyOrEnv("WEBDRIVER_GRID_BROWSER", "firefox");
String gridBrowserVersion = getPropertyOrEnv("WEBDRIVER_GRID_BROWSER_VERSION", "");
String gridBrowserPlatform = getPropertyOrEnv("WEBDRIVER_GRID_BROWSER_PLATFORM", "");

DesiredCapabilities gridCapabilities = new DesiredCapabilities();
gridCapabilities.setBrowserName(gridBrowser);
if(gridBrowserVersion.length()>0)
    gridCapabilities.setVersion(gridBrowserVersion);
if(gridBrowserPlatform.length()>0)
    gridCapabilities.setPlatform(Platform.fromString(gridBrowserPlatform));

And getPropertyOrEnv looks as follows


/**
 * Allow setting the controls via property or environment variable
 * property takes precedence, then environment variable, then default
 */
private static String getPropertyOrEnv(String name, String theDefault){

    String theValue = System.getProperty(name);
    if(theValue == null){

        System.out.println("Could not find Property " + name);
        theValue = System.getenv(name);

        if(theValue==null){

            System.out.println("Could not find Environment Variable " + name + " using default value " + theDefault);
            theValue = theDefault;

        }else{
            System.out.println("Using Environment Variable " + name + " with value " + theValue);
        }
    }else{
        System.out.println("Using Property " + name + " with value " + theValue);
    }

    return theValue;
}

This supported my local debugging, CI and console mvn test triggering.

But as with most tasks related to automating systems, I should expect to encounter additional maintenance and workarounds. And such was the case when I tried to execute the tests against BrowserStack in addition to Saucelabs and my local grid.

Basic Saucelabs configuration uses the capabilities:

  • browser
  • platform
  • browser_version

Basic BrowserStack configuration uses the capabilities

  • os
  • os_version
  • browser
  • browser_version

And of course, with BrowserStack we want it to take screenshots to track the @Test method execution so we need to set

  • browserstack.debug=true

Since I have a ‘generic’ grid config in my Driver.java I wanted to support the extra capabilities in a more generic way.

And I still wanted to allow them to be configured via environment variables or properties.

So, I run through all the environment variables and properties looking for any name prefixed with WEBDRIVER_GRID_CAP_X_

// Allow adding any capability defined as an environment variable
// extra environment capabilities start with "WEBDRIVER_GRID_CAP_X_"

// e.g. WEBDRIVER_GRID_CAP_X_os_version XP
// e.g. WEBDRIVER_GRID_CAP_X_browserstack.debug true
Map<String, String> anyExtraCapabilities = System.getenv();
addAnyValidExtraCapabilityTo(gridCapabilities, anyExtraCapabilities.keySet());

// Now check properties for extra capabilities
Properties anyExtraCapabilityProperties = System.getProperties();
addAnyValidExtraCapabilityTo(gridCapabilities, anyExtraCapabilityProperties.stringPropertyNames());

And the magic addAnyValidExtraCapabilityTo method, looks as follows:

private static void addAnyValidExtraCapabilityTo(DesiredCapabilities gridCapabilities, Set<String> possibleCapabilityKeys) {

    String extraCapabilityPrefix = "WEBDRIVER_GRID_CAP_X_";

    for(String capabilityName : possibleCapabilityKeys){

        if(capabilityName.startsWith(extraCapabilityPrefix)){

            String capabilityValue = getPropertyOrEnv(capabilityName, "");

            if(capabilityValue.length()>0){
                String capability = capabilityName.replaceFirst(extraCapabilityPrefix,"");
                System.out.println("To Set Capability " + capability + " with value " + capabilityValue);
                gridCapabilities.setCapability(capability, capabilityValue);
            }
        }
    }
}

Nothing earth shattering there, but it allowed me to add flexibility into the config without adding too much extra code into the Driver.java class.

If I was doing this in the real world. I would have created a new configuration reader type object, rather than adding the getPropertyOrEnv into the Driver.java code, but currently, the only config I use relates to the Driver.java code and I try to avoid too many radical code changes to the course if I can avoid it.

Also, in the real world, you tend not to need to add such a large amount of flexibility. You tend to stick to one grid provider, or refactor your code into more specific abstractions.

So don’t take this code as an exemplar of how to configure your remote driver, instead look upon it as a set of quick, but pragmatic changes to add more flexibility into the code base, and perhaps you’ll see something you can re-use here.

I’ve added additional lectures into my course to explain the config I used for Saucelabs, BrowserStack and TestingBot. And uploaded all the code changes extracted here to the code in the Selenium WebDriver with Java course.

Posted in Grid, Java, WebDriver | 2 Comments

FAQ: Why has my Firefox Selenium WebDriver Browser Not Closed

A few FAQs come up on the Start Selenium Simplified course. When they do, it usually means that I haven’t done a good enough job of making the answer visible the first time I address it.

One that crops up a lot is “Why is my browser not closing?”

Because WebDriver has two methods:

  •  .close
  • .quit

The documentation for .close says that it will close the browser if it is the last window that is closed.

For the last few Firefox releases, this does not happen.

Therefore we use .close, and .quit.

Use .quit when you know it is the last window you are working with. Or if you are working with a Driver Manager abstraction then you might want to stick the .quit in a JVM Shutdown.

I don’t know the reasons why, I haven’t spent the time investigating. But the workaround is to use .quit to finish with the driver.

 

Posted in FAQ, Firefox, Java, WebDriver | Leave a comment

FAQ: Why are my JUnit asserts showing as deprecated

I tend to use Hamcrest’s AssertThat now, but I still write Assert in some of my example code.

JUnit has a tendency to move code around, but it deprecates the old code, and there is usually a comment in the JUnit code somewhere explaining what to use instead.

But a common question on the Start Selenium WebDriver course relates to “Why are my asserts deprecated” and its usually because JUnit codebase has changed.

These changes are usually for the better, so keeping our code up to date and changing the import statements is a small price to pay.

Short Answer? Use the import from org.junit.Assert

 

Posted in FAQ, Java | Leave a comment

What do Eclipse, JUnit, Maven, Ant, etc. do?

I periodically receive FAQ style questions. But I haven’t created an FAQ page… yet.

Recently there have been an influx of people on the free “Start Using Selenium WebDriver With Java” course, and they are a little confused by the amount of software they install.

Now the course does explain a little about what the software does when using them but the course hasn’t provided an overview… until now.

The text summary follows below the video:


We have some mandatory installs:

  • Selenium WebDriver
  • Java JDK
  • Firefox
  • JUnit

We install the Java JDK because we are writing and compiling Java code.

We install Firefox because Selenium WebDriver comes bundled with all the necessary drivers for Firefox so that becomes our default first browser to automate because Selenium WebDriver makes it easy to get started with Firefox out of the box.

We have to install Selenium WebDriver, otherwise we can’t automate anything. And when we use Maven, we don’t actually need to install it, we define it as a dependency, but if we use Ant we have to do additional work to install it.

We have to install JUnit because we use that as our default Test Runner and without it we can’t run any tests. And again, when we use Maven, we don’t actually need to install it, we define it as a dependency, but if we use Ant we have to do additional work to install it.

The course has some Optional installs. You can follow the course completely without these. But they make it easier for me to work in windows, and make it clearer when I’m recording the videos, the actions that I’m doing.

  • Command Line Replacements instead of cmd: Console2 and ConEmu
  • Rapid Environment Editor instead of the default Windows Environment property editing dialog, which can be hard to see what changes have been made.

These are completely optional, and you don’t need to install them.

We need some sort of build system. Either:

  • Ant
  • Maven

You choose one, and only need to install one. I recommend Maven.

With Maven, you declare WebDriver and JUnit as dependencies in the project configuration file, the pom.xml, and maven will download those dependencies for you.

With Ant, again you have to declare the dependencies in a build.xml configuration file. But you also need to download and install Selenium WebDriver, and JUnit.

So maven makes it easier when you are getting started as you have a few less things to worry about getting right.

You need to install one of the IDEs:

  • IntelliJ
  • Eclipse

I recommend IntelliJ as people seem to have fewer out of the box issues with IntelliJ.

The Eclipse install and setup can vary between machines and beginner’s don’t need the additional hassle of trying to figure out why things aren’t working.

IntelliJ seems to work out of the box more easily.

So, your choice:

  1. Optionally install the helper tools,
  2. Install a JDK,
  3. Install Firefox,
  4. Choose which build system you want Ant, or Maven
    1. If you choose Ant then you will need to download Selenium WebDriver and JUnit
    2. If you use Maven, then you will configure maven to download Selenium WebDriver and JUnit for you
  5. Choose and install one IDE (I recommend IntelliJ)

Hopefully this helps people only follow the parts of the introductory course that they need to.

 

Posted in Courses, FAQ, Java, WebDriver | 4 Comments

New Book “Java For Testers”

Most of the Selenium WebDriver work I do uses Java.

When I wrote the “Selenium Simplified” book, I was trying to teach Java, at the same time as Selenium-RC.

I decided that the next time I write a book I would be more focused, so I’ve just made public the fact that I’m writing a book called “Java For Testers”.

You can find more details on the book web site’s and blog:

The book is in beta, so I’m still writing it. This also means that if you buy it early you can get hold of it at a reduced price. The price will increase the closer to production ready it becomes.

Posted in Java | 4 Comments