I had to make a few other minor changes which I summarise below.
Chrome and Firefox
Chrome and ChromeDriver 2.16 now seem to work as well as the Firefox driver. I used to have some issues with Frames and Window management but that has all gone away. Now I have a single set of tests for Chrome and Firefox with no ‘fails on Chrome’ and ‘fails on Firefox’ suites.
I did have to add some additional synchronisation when creating new windows in Firefox. Firefox used to block before returning control to the code when creating new windows, but now it doesn’t so I simple ‘wait’ for the element I need to be available before clicking on it. General synchronisation goodness that we all know and love.
I did have to make changes for cookies though. I assume most people don’t mess much with cookies when using WebDriver, but it is handy functionality when you need it.
ChromeDriver still has a bug where it creates 2 cookies. And now, when ChromeDriver creates a cookie it prefixes the ‘domain’ with a ‘.’. This means you have to be careful working with cookies through ChromeDriver as you might have to work around the browser creating a duplicate cookie after you amend it.
Firefox updates have meant that I now have to append a ‘/’ to my cookie paths.
The Select support class gave me a string padded text, so I had to ‘trim’ the output. I suspect this was a browser compatibility thing with my app code.
On Mac, my window positioning code didn’t work because I set the Y value to 20, but Mac wanted more space for its top menu bar, so I changed the position to 40.
That stopped when Opera went from version 12.17 and the Opera Driver didn’t work on most recent versions of Opera when they moved to webkit and then blink. But if you still wanted to, you could still use the driver and run tests with the older version of Opera.
But no longer.
The OperaDriver hasn’t been updated so it now compile clashes with Selenium WebDriver 2.46.0
You can exclude the conflicting apache.commons code if you want:
<!-- changes to Selenium 2.46.0 mean that the operadriver
now conflicts with Selenium WebDriver. I excluded the
apache.commons commons-exec from operadriver to allow
the driver to retain compile time compatibility
That will get you compiled, but won’t help you execute.
In fact OperaDriver hasn’t executed since WebDriver version 2.34.0
Since the Selenium API has been deprecated by the main WebDriver project, although it can still be found in the current versions of Selenium WebDriver, I have moved the example to a separate github project.
I’ve been an automator and a test manager so I’ve experienced both sides of the coin, but I’ve always had the benefit of technical and programming knowledge and experience. So while I had an initial list of areas that I thought current managers working, or about to work, with automation would want to discuss, I wanted to see what would happen on the day.
At this point I’m working from memory without notes, because I was facilitating a discussion, so let’s see what I remember…
The Test Management Summit is a discussion based event, with the emphasis on the presenter moving to triggering and facilitating discussion rather than doing all the talking. As a result, when you look at the slides, the first half of the slides were designed for me to talk over, and the remainder held in reserve in case the discussion faltered.
In the event, I talked for 15 mins, and the discussion was lively and proceeded without requiring my reserve slides.
I made a distinction between use of tools, and automation. With automation having the characteristic of no manual intervention during its execution.
I emphasized the need to have an understanding of how the automation works. If you don’t understand how it works then you have added a dependency on your success on someone who does. This applies across the board:
If you use a commercial tool then you rely on the support team from the supplier
If you use open-source then you rely on the internet and the informal development team, but have the opportunity to involve yourself, if you have the skills
Similarly if you bring in frameworks or libraries into your automation, if you don’t have the skill sets to read, and possibly amend the code, then you limit the level that you can understand or fix them.
Regardless, we have to have some sort of way of deciding if the automation process supports your test approach, or not.
As to “Who can use automation?”. If you don’t have the skills to move beyond automation as ‘magic’ then you can only use it if an interface has been added to the automation which allows your skill level to engage with it. i.e. automation with one button to start it can be triggered by anyone, but if that is the level of your understanding then you won’t have the ability to intervene when something goes wrong.
This may, or may not, be managed as a risk in your organization.
Many people do not think through what they want from automation to identify if their approach to automation can meet their expectations, or to think through multiple approaches to achieving their aims. Too often we default to: “automate all the scripts”. I recommend thinking through your aims, and identifying options. This will also help you work out if your automation ‘works’ i.e. helps you meet your aims.
Carl Dreher’s book is out of print, but I like it, as an historical overview of various forms of automation from mechanical devices through to cybernetics and computing. There are various quirks in language that I enjoyed, particularly the use of “Automationist” rather than “Automator” to describe someone who works on automation.
I also referenced “The Art of Leadership” by Captain S.W. Roskill. Another out of print book, but I can relate to the early sections in Roskill’s book where he describes two traits of military leaders that we don’t always expect from leaders in business:
An expectation that military leaders will train and teach as they gain experience
An expectation that military leaders need technical skills to ‘lead’ from the front and gain the respect of their troops
In order to balance this out for the business world, I think we make a distinction between ‘managers’ and ‘leaders’. Therefore managers may need to engage with automation in a different way than leaders who exhibit the traits above.
Should we convert manual scripts to automation scripts?
What would you do in the event of …
Single staff member having all the knowledge
Immediate “No – can’t be done” to requests for work
Staff who are ‘doing their job’ but are finished in half the day
How do you know your automation is working
How to recruit technical staff
What should a test manager do to improve their technical knowledge
What should a test programme manager do to improve their technical knowledge
Much of the discussion will remain private because it was provided by the participants themselves.
As add on points – during the discussion, I remember mentioning:
Use multiple layers of abstraction to support:
different “Who can use?” levels
use of automation in different ways, i.e. you can use the automation abstraction as a tool to support your testing, rather than only create non-manual intervention automation
Recruit with hands on exercises, i.e. pairing, discussion
Automating manual scripts does not lead to effective automation because
does not harness the ability for data driven automation (vary non-path invariant data)
does not harness the ability to use random data to explore assumptions inherent in equivalence class analysis
building on a point made by Adam Knight [http://www.a-sisyphean-task.com/] it does not encourage ‘alternative’ thinking about how automation can help test the system in ways that the script analysis did not identify e.g. API testing, adhoc automation to support defect investigation
Coaching without technical knowledge
Ask open questions of the team (i.e. non yes/no answers)
Ask “How can we…” questions
Encourage alternative investigations, but constrain them to time or scope
I added the notes and video for my ‘speedrun’ install of WebDriver 2.43.1 with Java, Maven and IntelliJ on the speedrun page.
I used VM Fusion to create a VM on my Mac with a clean install of OS X 10 Mavericks, then went through the process of installing on the Mac.
I updated the speedrun checklist to account for the Mac OS differences, and made notes on the install.
For some reason, when I installed on my ‘proper’ mac, I didn’t have to set the JAVA_HOME variable, but I did on the VM, so the video shows a full environment setup for JAVA_HOME, M2_HOME, M2 and updating PATH.
I found it interesting how Mac prompted me to the correct location a few times – ie. the install of JDK 1.8, so I did this earlier in the process than I did with Windows XP.
Also, since Mac OS X 10 Mavericks is slightly more ‘modern’ than Windows XP, I didn’t have to install an alternate browser to download the files, or an alternate text editor to amend the pom.xml file and update it to version 2.43.1 of WebDriver.
I edited the video down to 13 minutes, although the elapsed time (if I exclude the Wifi failure I had in the middle, was about 45 minutes).
You can find the speedrun video and installation notes on the speedrun page.
I confess to some nervousness about releasing the information since I don’t normally release the slides to tutorials and courses in case it cuts down on the value of running the tutorial or course again.
But, in this case, I don’t see enough information on the web about multiple Abstraction Layers and different ways of doing Page Objects, so I thought I’d throw my work out there, along with the supporting source code.
On a related note. I fronted a discussion at the Test Management Summit 2014 on Automation Abstractions. The slides for which were taken from the above presentation.
The aim for the slides and presentation is to demonstrate multiple approaches, so that people don’t just pick up the first ‘framework’ they see, or build ‘Page Objects’ without thinking through the modelling approaches open to them.
In the code and presentation I have examples of:
Page Objects that synchronize as slow loadable components
Navigation separate from Page Objects
DOM WebElement abstractions
The code and slides demonstrate some of my biases.
The main point is that ‘none of these approaches are best’, and we make decisions when we build our automation. We should take responsibility for those decisions and experiment with what works best in our environment, with our coding skills, with our development standards, for our application.
I’ll expand this material in the future. But I hope it helps somewhat in its current form.
You might also find the slides for my BDD tutorial relevant since I discuss some of my thoughts on BDD and Domain Specific Languages as Abstraction Layers.