Category Archives: Practices

Coding for workarounds so you know when they are fixed

With most libraries you use to automate your work, you have to code workarounds.

I coded a workaround in my Selenium WebDriver tests because there was a bug in ChromeDriver which meant that cookies were created differently than other browsers.

When I upgraded to WebDriver 2.53.0 and ChromeDriver 2.21 I discovered I didn’t need my workaround anymore.

In the code I had one set of code for Chrome, and another set for other browsers:

// chromedriver has a bug where the domain is prefixed with '.' 
//therefore
// the app 'updates' a new cookie so when chromedriver
// returns the cookie it returns
// the first and not the updated one
if(Driver.currentBrowser()== Driver.BrowserName.GOOGLECHROME){
  aCookie = getCookieWithLargestValueNamed
               ("seleniumSimplifiedSearchNumVisits");
}else {
  aCookie = driver.manage().getCookieNamed(
                "seleniumSimplifiedSearchNumVisits");
}

The bug was 7499 and was fixed in the recent update to ChromeDriver.

How do I know it was fixed?

Because I had failing @Test methods.

Tests in error:
  changeCookieVisitsCountUsingCookieBuilder
  (com.seleniumsimplified.webdriver.cookies.
   CookiesExercisesTestWorkWithExtraSync): 
  Chrome cookie creation issue has been resolved
  changeCookieVisitsCount(
    com.seleniumsimplified.webdriver.cookies.
    CookiesExercisesTestWorkWithExtraSync): 
  Chrome cookie creation issue has been resolved

Why did they fail (the workaround was still valid)?

Because I had code that checked for that.

// This currently fails on Chrome Driver since
// ChromeDriver adds some extra cookies when it creates 
// a cookie with a domain
// https://code.google.com/p/selenium/issues/detail?id=7499
if(Driver.currentBrowser() != Driver.BrowserName.GOOGLECHROME){
  assertEquals("we added the cookie correctly", cookieCount, 
               driver.manage().getCookies().size());
}else{
  if(driver.manage().getCookies().size()==cookieCount){
    throw new RuntimeException(
                 "Chrome inderal online order cookie creation issue has been resolved");
  }
// also Chromedriver now adds a '.' in front of the domain
}

I added extra code to throw an error, if the problem I was coding a workaround for went away.

I don’t know if that is a normal practice. But since I don’t really want workarounds in my code forever, and I likely won’t remember to check the release notes on every fix to see if I coded a workaround for a fix. This seems like a good workaround to check if workarounds are no longer good.

The code I was using to check for the largest value in a cookie, by the way, was:

private Cookie getCookieWithLargestValueNamed(String cookieName) {
    int largestCookieVal=0;
    Cookie largestCookie=null;
    for(Cookie aCookie : driver.manage().getCookies()){
        if(aCookie.getName().contentEquals(cookieName)){
            int cookieVal = Integer.parseInt(aCookie.getValue());
            if(cookieVal>largestCookieVal) {
                largestCookieVal = cookieVal;
                largestCookie = aCookie;
            }
        }
    }
    return largestCookie;
}

 

Posted in Practices, Selenium Simplified Blog, WebDriver | 5 Comments

Test Automation in Cybernetics And Management

I recently re-read Stafford Beer’s 1967 “Cybernetics and Management”. An overview of Cybernetics and its derivation from Operational Research; because its examples use organisations and teams, it provides immediate relevance to my work.

I re-read this because I like to understand ‘automation’ from a more historical perspective, rather than focus on it from the perspective of ‘test automation’.

An Unfortunate Word

A paragraph towards the end of the book stood out:

“… Automation, in short, has turned out to be an unfortunate word, because it seems to say that there is something there to be made automatic. That is to say “automation” seems to imply that we know what to automate, which is by no means true.

I have come to view ‘Automation’ as “an unfortunate word” and in some of my Keynotes fron 2015, I recommended avoiding the use of the ‘automation’ word altogether.

As a word, ‘Automation’ can be interpreted as:

  • a process of automating something, and
  • a thing in its own right

… possibly other interpretations that I can’t think of at the moment.

Automation as a process

The process of automating something doesn’t present as many problems when communicating if I avoid the ‘automation’ word.

I could say:

“I am writing code to execute ‘path X’ through the system using the data set Y and check conditions in the set Z”

And that fairly clearly summarises my current task (if we know what ‘X’,’Y’ and ‘Z’ refers to).

If I replace ‘automating’ with ‘automation’, then I communicate with more ambiguity:

  • “I’m doing test automation” (automation as a verb)
  • “I’m creating test automation for path X”, (automation as a noun)
  • “I’m writing test automation” (automation as a noun)

I could say:

“I am doing test automation by writing code to execute ‘path X’ through the system using the data set Y and check conditions in the set Z”

And if I do, I probably inserted ‘test automation’ as a marketing phrase because I think the listener will zone out when I start talking about the actual details of my task.

Against Automation as a Thing

We can count Things. We can cost Things.

This leads to ‘test automation’ as a metric-able, ‘return on investment’-isable, management obfuscation-isable concept.

Which has a tendency to encourage work and discussions mired with red tape.

If we avoid the word ‘automation’ then we can avoid much of this.

What other words can we use?

Norbert Weiner wrote a paper called “Some Moral and Technical Consequences of Automation” where he uses the word ‘Automation’ in the title.

A good copywriting technique involves using words that your readership understands, so that they then read the next sentence in the article, and keep doing that to build an effortless flow through the article. A technique I have yet to consciously exploit.

In the body of the article itself Weiner uses words such as: program, automata, automate, automatization, strategy, machine, automatic, programming, etc.

Words which are very specific in their meaning, particularly when read in context.

Weiner avoided writing ‘Automation’.

We know what to automate

And back to Stafford Beer,:

‘automation’ seems to imply that we know what to automate, which is by no means true

The conjoint concept ‘test automation’ has the issue that because it relates to ‘test’ we might look at the visible portions of ‘test’ and then ‘automate’ what we see or have the ‘automation’ done by the people who ‘test’.

Rather than looking at the system of ‘testing’ and automating to improve or change the system.

We can automate to expand our possibilities rather than simply automate existing parts of the sub-system and then work around those.

Earlier in the same book

On pages 214 to 217 Stafford Beer describes many of the problems that I see written of today as ‘Test Automation Issues’

“The machines worked very fast, and looked like replacing large numbers of clerical staff – consequently the costing exercises indicated a favourable outcome for the machine. In the event, the staff savings were normally over-estimated; and the typical picture become one of disappointed managers finding that, after very intense effort and many teething troubles, they had a quired a new procedure which was not much better than the old – and certainly not much cheaper.”

Hmm, I think I’ve seen this written in terms of a ‘Test Automation’ issue.

“.. is is common nowadays that large companies boast about the *number* of computers they have… one looks in vain for accounts of the astounding new modes of control which have been made possible…”

Because ‘things’ are countable. And I view having more ‘things’ as better than having fewer ‘things’.

“The fact is that we should not have been trapped by our cost management technique into comparing existing procedures with their possible automated replacements. Because the exiting procedures set out to solve problems which arose in a computerless world, and those problems were generated by an organisational structure designed for a computerless world.”

A Knockout Punch

On page 217 Stafford Beer hits us with an uppercut to floor us:

“How can I use the computer in my enterprise?” … the question was wrong. A better question would have been “What should my enterprise be like, now that computers exist?”

It can take experience to answer this because you may not know the possibilities that have been opened up.

But if you have conducted experiments with various tools and approaches then you can start to expand your view of how automating might help you adapt your test process the the future variety in the System of Development.

And an easy first step? Stop using the word ‘automation’.


 

Like most Stafford Beer books “Cybernetics And Management” tends to sell at inflated second hand prices, but you can usually find affordable copies if you know where to look

References:

Posted in Practices, Selenium Simplified Blog | 4 Comments

Automation Abstractions: Page Objects and Beyond – Conference Talk

I presented “Automation Abstractions: Page Objects and Beyond” as a conference talk at Star West 2014 and Oredev 2014

Essentially the same talk, but Oredev had a slightly smaller timeslot for the talk, so I summarised the verbage a little.

I have uploaded the slides to slideshare

And the source code is on github.com/eviltester/automationAbstractions

The talk was filmed and recorded by Oredev, unfortunately the audio doesn’t appear to have recorded properly and is very quiet in their released vimeo video..

However, I have subsequently amended their video and re-processed the audio so you can hear it, and released it on YouTube and you can watch it below.

Posted in Conferences, Practices, Selenium Simplified Blog | 13 Comments

Selenium WebDriver Page Object Abstractions and Beyond

I’ve been collating my thoughts on Page Objects and Abstraction Layers, particularly the implementation of them using Java.

I presented my initial thoughts at Let’s Test 2014 in a half day workshop. This could easily have been a full day workshop to allow for more experimentation with the different coding styles.

I released the slides to the tutorial on slideshare.

I confess to some nervousness about releasing the information since I don’t normally release the slides to tutorials and courses in case it cuts down on the value of running the tutorial or course again.

But, in this case, I don’t see enough information on the web about multiple Abstraction Layers and different ways of doing Page Objects, so I thought I’d throw my work out there, along with the supporting source code.

I’m presenting this material as a ‘talk’ rather than a workshop at StarWest 2014, and at Oredev 2014.

On a related note. I fronted a discussion at the Test Management Summit 2014 on Automation Abstractions. The slides for which were taken from the above presentation.

The aim for the slides and presentation is to demonstrate multiple approaches, so that people don’t just pick up the first ‘framework’ they see, or build ‘Page Objects’ without thinking through the modelling approaches open to them.

In the code and presentation I have examples of:

  • Page Objects
  • Page Objects that synchronize as slow loadable components
  • Navigation separate from Page Objects
  • Domain Objects
  • DOM WebElement abstractions
  • and more…

The code and slides demonstrate some of my biases.

The main point is that ‘none of these approaches are best’, and we make decisions when we build our automation. We should take responsibility for those decisions and experiment with what works best in our environment, with our coding skills, with our development standards, for our application.

I’ll expand this material in the future. But I hope it helps somewhat in its current form.

You might also find the slides for my BDD tutorial relevant since I discuss some of my thoughts on BDD and Domain Specific Languages as Abstraction Layers.

Posted in Conferences, Practices, Selenium Simplified Blog | 4 Comments

Helper Classes for SlowLoadableComponent Page Objects

I generally caution against ‘Helper’ classes and Static Helper classes when I’m consulting for automation.

e.g. PageObjectHelper, or ApplicationHelpers, or StringHelpers etc.

Because ‘generally’, these ‘Helper’ objects mean “We created a class to dump stuff in because we couldn’t figure out how to model our abstraction layers” and often, people then dump more and more methods into them such that they become an undisorganized and unmaintainable mess.

Just so you know where I’m coming from here.

WebDriver has a ‘good’ Helper class in the form of the ExpectedConditions class. Every static method on it returns an expected condition which I can then use in a WebDriverWait, and these expected conditions represent very common wait conditions that I use across many projects.

Since I’ve started using the SlowLoadableComponent approach for most of my Page Objects recently, I have written a lot of code that looks like this:

    @Override
    protected void isLoaded() throws Error {
        try{
            if(!driver.findElement(By.id("field")).isDisplayed()){
                throw new Error("field is not displayed");
            }
        }catch(WebDriverException e){
            throw new Error("field is not displayed", e);
        }
   }

And it starts to make my Page Objects look pug ugly.

I have started to take a leaf out of the ExpectedConditions class rule book and have been creating an IsLoaded ‘helper’ Class

I have used 3 main approaches for this:

  • A Fluent Class
  • A set of Static methods
  • Combined Fluent And Static

In WebDriverExperiments on GitHub I have added a basic propranolol online canadian pharmacy ‘fluent’ class which I would use like this:

    @Override
    protected void isLoaded() throws Error {

       IsLoaded.forThis(driver).
               whenElementIsVisible(SEND_BUTTON, "Send Button").
               whenElementIsEnabled(SEND_BUTTON, "Send Button").
               whenElementIsVisible(NAME_FIELD, "Name Input").
               whenElementIsEnabled(NAME_FIELD, "Name Input");
    }

You can see I create an IsLoaded object via a factory method and then have the try/catch/’throw error’ stuff abstracted away on the IsLoaded methods.

Sometimes I make these static so the call would look more like this:

IsLoaded.whenElementIsVisible(driver, SEND_BUTTON, "Send Button");

And sometimes I mix it up so I can call Static methods, or use the fluent style on the above object.

I might even write code that allows me to write the following:

IsLoaded.forThis(driver).
  and().thisElement(SEND_BUTTON, "Send Button").
        whenElementIsVisible().
        whenElementIsEnabled().
  and().thisElement(NAME_FIELD, "Name Input").
        whenElementIsVisible().
        whenElementIsEnabled();

The above is just a thought experiment, but I haven’t been annoyed enough with the repeated parameters in the code I’ve uploaded to github to pursue it yet.

I think that this ‘Helper’ object is well enough named, and tightly enough scoped that it won’t cause a problem, so it doesn’t trigger my ‘Ugh, helper Class’ response.

I think a few more production usages and I’ll settle on a style that I’ll use as a default. At the moment, the ‘non-static’ approach is winning, hence no static example in the github code.

But what do you think? Leave a comment if you have an alternative modelling approach.

Source:

 

Posted in Practices, Selenium Simplified Blog, WebDriver | 2 Comments

A Fluent Page Object Approach – return this

Another simple tip, but I use this a lot.

Return ‘this’ from your Page Object methods

 

e.g.

QueNessSlowLoadingExamplePage page4 = 
       new QueNessSlowLoadingExamplePage(driver);
 page4.get();

page4.
 setName("Bob Dobbs").
 setEmail("b.dobbs@mailinator.com").
 setMessage("Hello There").
 sendMessage();

In the above code I have a Page Object called QueNessSlowLoadingExamplePage which handles this example over at QueNess.com

And you can see that the setName, setEmail and setMessage are all ‘fluent’ in the sense that they return ‘this’ so I can chain them when using the Page Object.

e.g.

public QueNessSlowLoadingExamplePage setName(String name) {
 driver.findElement(NAME_FIELD).sendKeys(name);
 return this;
}

public QueNessSlowLoadingExamplePage setEmail(String email) {
 driver.findElement(EMAIL_FIELD).sendKeys(email);
}

On returning ‘other’ Page Objects

Note that this does not mean that I advocate returning ‘other’ Page Objects from a Page Object

i..e if ‘sendMessage’ causes a chain of system events such that a new page is displayed in the browser then I don’t advocate returning that page as a Page Object from the ‘sendMessage’ object.

If navigating to a new page was a possibility I would make ‘sendMessage’ a ‘void’ object, because I treat ‘Navigation’ as a separate responsibility, managed by a separate set of Objects.

On returning child components

Note that this does not mean that I never return ‘child’ or ‘container’ Page Objects.

e.g. if this page had a Widget, that I had modelled as a separate Page Object, I am quite happy to return a ‘new Widget’ Object from a method on this Page Object, because this Page Object would be acting as a container for that one.

Not an absolute rule

But not navigation.

Not for me.

At least, most of the time, because all rules are made to be broken, thereby granting us the requisite variety we need to model our applications effectively. So sometimes I may choose to navigate via Page Objects. Very often I do this for expediency, or for throw away code. I’ll very likely refactor it out later.

You can find code that illustrates this on github in my WebDriverExperiments repository.

Is this what you do?

Is this what you do? Leave a comment if you have any other Page Object Method tips.

Posted in Practices, Selenium Simplified Blog, WebDriver | 9 Comments

How to write a keyword driven test framework – an historical perspective

I don’t really like keyword driven frameworks. For a whole bunch of reasons that I won’t go into now, since you came here to learn how to write them.

Well, here’s the secret…

This is old tech. Older than old.

In fact, when I see it being promoted as ‘state of the art’ I’m amazed, and a little disappointed.

The first book I read that relates to this was “Write your own adventure programs for your Microcomputer”, back in 1983, and it teaches the basics of how a keyword driven framework works. I was using a ZX Spectrum 48K at the time. I loved these little Usborne books, they probably set me on the road to programming better than any other resource.

I don’t think I managed to get the listing in the book working, but I did manage to write my own crude adventure game parsers.

In fact, pretty much any tutorial on coding text adventure games will cover the basics.

And this is partly why Keyword driven frameworks exist.

Keyword Driven Frameworks exist because they are easy to write.

Not because they boost productivity, or are easy to maintain or anything like that. Simply because they are easy to write.

And as much as I denigrate them, if you write your own, then you may well learn lessons about parsing input that will help you improve your test automation abstraction layers in the future, so I shouldn’t be too hard on them.

And yet, I caution you to be cautious.

The basic principles you have to address for Keyword Driven Testing are:

  1. Decide on a vocabulary and phrases
  2. Decide on a parsing mechanism
  3. Decide how to map your custom code to the verbs you parsed

Decide on a vocabulary and phrases

 With an old text adventure you used a basic Verb/Noun parser. e.g.

  • Take Book,
  • Open Door

We might relate that to:

  • Open_Browser Firefox
  • Goto_URL http://www.eviltester.com
  • Assert_Title “Home | Evil Tester.com”

One of your first tasks is to decide on the Verbs you want to handle in your application and your domain.

 Decide on a parsing mechanism

Then you have to decide how you are going to parse them. csv? tab delimited? plain text?

When writing text adventures you have the disadvantage that you have to handle user input, dynamically.

What we did in the old days, with a verb noun parser was split the string looking for the first space. Everything before the space was the verb, everything after the space was the noun.

And that’s pretty much all you do with keyword driven testing. You take the line of input and split it based on whatever delimiter you choose.

This is why a lot of people use Excel format as input, you use a library that reads Excel, and treat each cell as an input. The first cell on the line is the verb, the rest are the nouns or parameters.

You can get more complicated, and indeed, most home grown keyword driven test frameworks seem very crude given the type of parsing we eventually learned how to do when coding adventure games.

But if you want to do this, and you want to get started now, then keep it simple.

You might want to split your input verbs and nouns with tabs, then you can accommodate spaces in words and parameters. Up to you.

Decide how to map your custom code to the verbs you parsed

Assuming you have parsed the input and stored the verbs and nouns in some variables and some sort of collection, then you have to decide what to do with that data and map it on to custom methods.

There are lots of ways of doing this, and your level of programming experience will pretty much dictate the one you choose.

In the game above, they tokenised the input and associated each word with a number, to make it easy to decide what to do. This was also done because of memory constraints. Its actually a pretty good way of working, but most buy inderal 10mg people writing keyword driven frameworks don’t do this, they tend to do something like the following pseudo code:

If uppercase(keyword) = "OPEN_BROWSER" 
     then open_browser(params_array);
If uppercase(keyword) = "ASSERT_TITLE" 
     then assert_title(params_array);

And that’s fine, create a function for each keyword, then call it when you find it.

Job Done

And what do you know. You’ve done it. That’s it.

When writing an adventure game you have to maintain state, create maps, handle lots of objects, track location, handle random events, etc. etc.

But with a keyword driven test framework you just map a keyword to some code you’ve written and voila.

It’s ugly. It’s crude. It’s basic. And it’s still what many people do.

If you want to get ‘fancy’ then you could read a ‘how to write your own interpreter’ or compiler book. And there are loads of those. I’m not up to date on them because I studied this stuff years ago, although one of my favourites [Aho, Sethi and Ullman]’s “Compilers: Principles, Techniques and Tools” is still well regarded

Although I prefer the Dragons cover, which is also much more fitting with the text adventure theme.

I still have my copy of “The Dragon” book, it seems to be the only compiler/interpreter book I kept. (In addition I also have “Introduction to Automata Theory Languages and Computation” by Hopcroft and Ullman, which isn’t quite as simple to pigeon hole )

Why not use a simple and more CV friendly approach?

However, we have learned a lot now about Domain Specific Languages, and writing maintainable abstraction layers. So instead of writing a simple text adventure parser, why not use something like Cucumber.

Cucumber essentially implements the parsing for you, and the execution, and the reporting, and the running, and all you do are implement the methods which map on to the commands, e.g. the ‘open_browser’ method above.

Yes your code and scripts will remain pretty horrible because you’ll have abstracted at a verb keyword level rather than a logical action level, but that’s exactly what you were going to do with a keyword driven approach anyway.

At least this way, you get a head start with a ‘modern’ tool so you don’t have to re-invent the wheel.

And you can gradually refactor your test abstractions to something readable and maintainable if you start to see issues with a keyword driven approach.

This also has the benefit that it is in vogue, and therefore easy for other people to understand as an addition to your CV. No-one really cares if you can do verb/noun parsing anymore, but if you can use Cucumber, they will look more closely at your CV.

Warning: Dragons within

But… here’s The Thing.

I loved writing parsers. I did. I wrote text adventure game parsers, AI parsers, Natural Language parsers, Interpreters and Compilers.

I loved it, they were great fun.

I learned a lot.

And that is part of the danger.

The people writing the framework are going to have a blast.

The people using it, are going to create long unmaintainable scripts that are ugly, confusing and brittle, and all in the name of Keyword driven testing.

So I warn you.

Your techy testers, when just starting out, are going to recommend Keyword Driven approaches because:

  1. The ‘industry’ still loves to promote them
  2. The frameworks are fun and challenging to build from a technical perspective
  3. Your testers don’t need to learn anything technical, just a ‘small’ vocabulary of verbs (or so the promotional material goes)

I caution you. Here be Dragons.

PS: If enough people contact me then I’ll write a short keyword driven framework example in Java, just to show how easy it can be, and how crude and ugly I can make my code.

PPS: I’ve long since given away my copy of “Write your own adventure programs for your Microcomputer” but I found it again on the wonderful World Of Spectrum  as a pdf

Posted in Practices, Selenium Simplified Blog | Tagged | 7 Comments