EvilTester.com on EvilTester.com - Exploratory Testing, Automation, Development (Agile + Web) Hugo https://www.eviltester.com/ en-us Alan Richardson Sun, 31 Dec 2023 16:04:49 GMT Kubernetes kubectl cheat sheet notes https://www.eviltester.com/blog/eviltester/devops/some-kubernetes-kubectl-cheat-sheet-notes/ Wed, 15 Nov 2023 08:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/devops/some-kubernetes-kubectl-cheat-sheet-notes/ <p>A cheat sheet to remind myself the most important <code>kubectl</code> commands. If I do forget then I recommend the VS Code Kubernetes plugin and remember to use <code>--help</code> if you get stuck, start with <code>kubectl --help</code> .</p> <p>I had to use kubernetes again after not using it for about a month and I had forgotten everything I had learned before.</p> <p>So I rummaged through my previous notes and collated the commands I&rsquo;ve mostly used.</p> <p>To help them stick in my head I categorised them with natural language terms &ldquo;Where am I?&rdquo;, &ldquo;what is this doing?&rdquo;, etc.</p> <h2 id="official-cheet-sheet">Official Cheet Sheet</h2> <p>The official <code>kubectl</code> cheat sheet is much bigger than this, but I don&rsquo;t tend to use all the commands.</p> <p><a href="https://kubernetes.io/docs/reference/kubectl/cheatsheet/">https://kubernetes.io/docs/reference/kubectl/cheatsheet/</a></p> <h2 id="use-a-ui">Use A UI</h2> <p>Part of the reason I forget the commands is that I often just use a tool to access and navigate the environments.</p> <p>I&rsquo;ve been using K9s which is a terminal UI, it is mostly easy to use, although you have to remember <code>vi</code> style commands to type <code>:</code> prior to entering a command like <code>pods</code> or <code>namespaces</code> to see a list of pods or namespaces. And <code>/</code> to enter a search term to filter the displayed list.</p> <p>Today I experimented with the VS Code plugin</p> <p><a href="https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools">Visual Studio Code Kubernetes plugin</a></p> <p>That seemed much faster and once I got used to it, it felt a lot more natural to use:</p> <ul> <li>select the context from the tree view</li> <li>either select the namespace from the list or click the toolbar at the bottom and enter the details</li> <li>now both the context and namespace are set as the defaults</li> <li>find the pods, jobs, etc. in the <code>Workloads</code> branch</li> <li>then right clicking shows options like <code>get</code>, <code>describe</code>, <code>logs</code>, <code>terminal</code></li> </ul> <p>Also the logs view allows regex matching through an easy UI which means I didn&rsquo;t have to use <code>stern</code> to monitor the logs.</p> <p>If you&rsquo;re working with Kubernetes then I do recommend the VS Code plugin, with the warning that extensive use of UI tools tends to weaken your CLI muscle memory.</p> <h2 id="cheat-sheet">Cheat Sheet</h2> <p>My cheat sheet notes for commands are below:</p> <h3 id="where-am-i">Where am I?</h3> <ul> <li><code>kubectl config current-context</code></li> <li><code>kubectl cluster-info</code></li> <li><code>kubectl config view --minify</code></li> </ul> <h3 id="where-can-i-go">Where can I go?</h3> <ul> <li><code>kubectl config get-contexts</code></li> <li><code>kubectl get namespace</code></li> </ul> <h3 id="work-with-a-different-context">Work with a different context</h3> <ul> <li>remind me what that context is again? <ul> <li><code>kubectl get context a-context-name</code></li> </ul> </li> <li>ok use that one <ul> <li><code>kubectl config use-context a-context-name</code></li> </ul> </li> <li>or <ul> <li><code>kubectl config set current-context a-context-name</code></li> </ul> </li> <li>ok set this namespace to be the default <ul> <li><code>kubectl config set-context --current --namespace=a-namespace-name</code></li> </ul> </li> </ul> <h3 id="what-is-here">What is here?</h3> <ul> <li><code>kubectl get pods</code></li> <li><code>kubectl get jobs</code></li> <li>etc.</li> </ul> <p>use <code>kubectl api-resources</code> for a list of supported resources</p> <h3 id="what-is-_that_">What is <em>that</em>?</h3> <ul> <li><code>kubectl describe pod a-pod-name</code></li> <li><code>kubectl describe job a-job-name</code></li> <li>etc.</li> </ul> <h3 id="what-is-it-doing">What is it doing?</h3> <ul> <li><code>kubectl logs -f a-pod-name --follow</code> <ul> <li>follow tails it, so leave it off if you want a log dump</li> </ul> </li> <li><code>kubectl top pods</code> <ul> <li>watch the command <code>watch kubectl top pods</code></li> </ul> </li> </ul> <h3 id="how-do-i-access-that">How do I access that?</h3> <ul> <li><code>kubectl get ingress -n a-namespace</code></li> </ul> <h3 id="am-i-allowed-to-do-that">Am i allowed to do that?</h3> <ul> <li><code>kubectl auth can-i get pods -n a-namespace</code></li> </ul> <h3 id="how-do-i-run-commands-there">How do I run commands there?</h3> <ul> <li>get a shell <ul> <li><code>kubectl exec --stdin --tty a-pod-name -- /bin/bash</code></li> <li><code>kubectl exec -it a-pod-name -- /bin/bash</code></li> </ul> </li> <li>run a command directly <ul> <li><code>kubectl exec a-pod-name -- ls</code> <ul> <li>(add <code>--stdin --tty</code> for nicely formatted output)</li> </ul> </li> </ul> </li> </ul> <h3 id="how-do-i-delete-something">How do I delete something?</h3> <ul> <li><code>kubectl delete pod a-pod-name</code></li> <li><code>kubectl delete job a-job-name</code></li> <li>etc.</li> </ul> <h3 id="how-do-i-work-with-jobs">How do I work with jobs?</h3> <ul> <li> <p><code>kubectl get jobs</code></p> </li> <li> <p>get all job names that match a particular pattern</p> <ul> <li><code> kubectl get jobs --no-headers=true | awk '/part-of-name-.*/{print $1}'</code></li> </ul> </li> <li> <p>delete all job names (and pods) that match a particular pattern (add <code>xargs</code> to above)</p> <ul> <li><code>kubectl get jobs --no-headers=true | awk '/part-of-name-.*/{print $1}' | xargs kubectl delete job</code></li> </ul> </li> <li> <p>trigger a cronjob</p> <ul> <li><code>kubectl create job --from=cronjob/a-cronjob-name &quot;name-for-new-job-$(date +%s)&quot;</code></li> </ul> </li> </ul> <h3 id="common-patterns">Common Patterns</h3> <ul> <li>add <code>--help</code> to find out what a command does e.g. <code>kubectl auth --help</code></li> <li>add <code>-n a-namespace</code> to run a command in a namespace or <code>--namespace a-namespace</code></li> <li>add <code>-c a-context</code> to run a command in a namespace or <code>--context a-context</code></li> <li>add <code>-o json</code> to see the output as json (e.g. for piping into jq)</li> </ul> <h3 id="supporting-tools">Supporting Tools</h3> <ul> <li><code>kubectx</code> + <code>kubens</code> make it easy to switch context and namespace <a href="https://github.com/ahmetb/kubectx">https://github.com/ahmetb/kubectx</a></li> <li>stern makes it easy to tail pod logs <a href="https://github.com/stern/stern">https://github.com/stern/stern</a></li> <li>kubectl has a plugin eco-system which can be accessed using krew <a href="https://krew.sigs.k8s.io/">https://krew.sigs.k8s.io/</a></li> <li>k9s terminal UI for K8s <a href="https://k9scli.io/">https://k9scli.io/</a></li> <li>Visual Studio Code Kubernetes plugin <a href="https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools">https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools</a></li> <li>Jetbrains IntelliJ Kubernetes plugin <a href="https://plugins.jetbrains.com/plugin/10485-kubernetes">https://plugins.jetbrains.com/plugin/10485-kubernetes</a> (not CE compatible)</li> </ul>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Updated Github Repos and Released Conference Talk https://www.eviltester.com/blog/eviltester/news/github-updates-and-talk-recordings/ Sun, 30 Jul 2023 09:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/news/github-updates-and-talk-recordings/ <p><em>TLDR; Updated code for Selenium WebDriver Course for WebDriver v4 free online and released Open Source Profile conference video for free.</em></p> <h2 id="selenium-webdriver-code-updates">Selenium WebDriver Code Updates</h2> <p>When I removed my Selenium WebDriver course for sale and made it available to <a href="https://www.testerhq.com/member2/index.php?course=selenium_webdriver_3_api&amp;page=coursecontents">Patreon supporters only</a> this was because I didn&rsquo;t really have time to update the videos.</p> <p>I have however, been updating the <a href="https://github.com/eviltester/seleniumWebDriverAPICourse">source code</a>.</p> <p>The full code for the course has been updated to:</p> <ul> <li><a href="https://github.com/eviltester/seleniumWebDriverAPICourse/tree/main/code/selenium_3/code">Selenium WebDriver 3.141.59</a></li> <li><a href="https://github.com/eviltester/seleniumWebDriverAPICourse/tree/main/code/selenium_4/code">Selenium WebDriver 4.10.0</a></li> </ul> <p>The code now uses the styled pages at <a href="https://testpages.herokuapp.com">testpages.herokuapp.com</a></p> <h2 id="selenium-webdriver-test-pages-updated">Selenium WebDriver Test Pages Updated</h2> <p>Additionally I&rsquo;ve updated the Test Pages themselves.</p> <p><a href="https://testpages.herokuapp.com">testpages.herokuapp.com</a></p> <p>The old unstyled pages have been deprecated and are less obvious on the page. And I fixed up a few minor bugs on some of the pages.</p> <h2 id="selenium-webdriver-test-pages-running-locally-on-docker">Selenium WebDriver Test Pages Running Locally on Docker</h2> <p>I&rsquo;ve also added <a href="https://github.com/eviltester/TestingApp/blob/master/java/docker/selenium-test-pages/Dockerfile">config</a> to allow the test pages to be run via docker.</p> <p>After cloning the repo or downloading via zip from <a href="https://github.com/eviltester/TestingApp">github.com/eviltester/TestingApp</a></p> <p>From the &lsquo;\java&rsquo; directory run:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>mvn package </span></span></code></pre></div><p>Then:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>docker build -t seleniumtestpages <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span>-f ./docker/selenium-test-pages/Dockerfile . </span></span></code></pre></div><p>Then:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-shell" data-lang="shell"><span style="display:flex;"><span>docker run -p 4567:4567 seleniumtestpages </span></span></code></pre></div><p>This will make the test pages app accessible on <code>localhost:4567</code></p> <h2 id="open-source-profile-talk">Open Source Profile Talk</h2> <p>I noticed that the talk I created for the Open Quality Conference in 2020 was no longer available anywhere online.</p> <p>Full details about the talk are here:</p> <p><a href="https://www.eviltester.com/conference/openquality2020_conference/">Open Quality 2020 Talk Page</a></p> <p>The page above contains:</p> <ul> <li>talk outline</li> <li>blurb</li> <li>extra information</li> </ul> <p>I have now released that to YouTube.</p> <div class="video-container"> <iframe width="560" height="315" src="https://www.youtube.com/embed/_vDrAMDuN8w" allow="autoplay; encrypted-media" allowfullscreen></iframe> </div> <div><p class="center-text"><a href="https://www.youtube.com/watch?v=_vDrAMDuN8w" target="_blank">Watch on YouTube</a></p></div> <p>It had previously been released to <a href="https://www.patreon.com/posts/open-source-open-45556806">Patreon supporters ad-free</a> and with a full transcript, back in 2020.</p> <p>This has also been added to the <a href="https://www.testerhq.com/member2/index.php?course=get_your_message_out___part_1___blogging&amp;page=coursecontents">Patreon Exclusive Online Course &ldquo;Get Your Message Out - Blogging&rdquo;</a></p> <p>The <a href="https://github.com/eviltester/eviltester-talks/blob/main/open-quality-conf/2020/open-source-profile/slides-export.md">slides are available on Github</a> in the EvilTester Talks repo.</p> <p><a href="https://github.com/eviltester/eviltester-talks">https://github.com/eviltester/eviltester-talks</a></p> <p>I&rsquo;m in the process of combining a bunch of repos and removing older repos. I&rsquo;m also hunting around my hard drive to see what I haven&rsquo;t released and makings sure I have enough top level repos to cover the content.</p> <h2 id="combined-github-repos">Combined Github Repos</h2> <p>I&rsquo;m in the process of combining a lot of smaller disparate repos into high level repos to cut down on the number of repos I have to review and maintain and hopefully make the content more discoverable.</p> <p>I&rsquo;ve pulled together 20 or so repos into a WebDriver Compendium repo:</p> <p><a href="https://github.com/eviltester/seleniumWebDriverCompendium">github.com/eviltester/seleniumWebDriverCompendium</a></p> <p>This contains smaller example repos and experiments.</p> <p>It also contains the Automation Abstractions repo which was the precursor to my Linkedin Course on Page Objects. <a href="https://www.linkedin.com/learning/advanced-selenium-page-objects-and-gui-automation">Advanced Selenium: Page Objects and GUI Automation</a></p> <p>The full source is available, as are the <a href="https://github.com/eviltester/seleniumWebDriverCompendium/blob/main/automationAbstractions/docs/abstractionLayers-slides.md">slides for the associated one day workshop</a>.</p> <p>I&rsquo;m still working through this repo to tidy and update the code, and as I do so I&rsquo;ll remove the earlier duplicated repos.</p> <h2 id="updated-getting-started-repos">Updated Getting Started Repos</h2> <p>I&rsquo;ve updated the &ldquo;Getting Started with&rdquo; repos.</p> <p>JUnit:</p> <ul> <li><a href="https://github.com/eviltester/startUsingJavaJUnit">Start Using Java Junit 4</a></li> <li><a href="https://github.com/eviltester/startUsingJavaJUnit5">Start Using Java Junit 5</a></li> </ul> <p>WebDriver with Java:</p> <ul> <li><a href="https://github.com/eviltester/startUsingSelenium4Java">Start Using Selenium WebDriver 3</a></li> <li><a href="https://github.com/eviltester/startUsingSelenium4Java">Start Using Selenium WebDriver 4</a></li> </ul> <h2 id="removing-travis-ci-and-running-webdriver-via-github-actions">Removing Travis-CI and Running WebDriver via Github Actions</h2> <p>I&rsquo;ve been meaning to migrate all the repos from Travis-CI to Github Actions since Travis-CI stopped offering free CI Builds for open source.</p> <p>I&rsquo;ve only now started upgrading.</p> <p>The JUnit repos and WebDriver 4 repos all use Github Actions to run the tests on push and via cron job.</p> <p>The new Selenium WebDriver 4 Browser Manager, makes running headless tests in CI much easier.</p> <ul> <li>see <a href="https://www.selenium.dev/blog/2022/introducing-selenium-manager/">Introducing Selenium Manager</a> on the official Selenium.dev site</li> </ul> <p>e.g. the <a href="https://github.com/eviltester/seleniumWebDriverAPICourse/blob/main/.github/workflows/build.yml">build.yml</a> for my Selenium WebDriver 4 course code is very simple:</p> <pre tabindex="0"><code>name: build on: push: workflow_dispatch: schedule: - cron: &#39;0 1 * * 6&#39; # weekly check that it still works env: BROWSER_STATE: Headless jobs: compile-and-test-webdriver-4: runs-on: ubuntu-latest steps: # https://github.com/actions/checkout - name: Download repository uses: actions/checkout@v3 # https://github.com/actions/setup-java - name: Set up JDK 18 uses: actions/setup-java@v3 with: distribution: &#39;temurin&#39; java-version: 18 - name: setup chrome uses: browser-actions/setup-chrome@v1 - run: chrome --version - name: Test with Maven using Chrome working-directory: ./code/selenium_4/code run: mvn test -Dselenium2basics.webdriver=CHROME </code></pre><p>This has:</p> <ul> <li><code>push:</code> to run the tests on push to the repo,</li> <li><code>workflow_dispatch:</code> to allow me to manually trigger the execution from Github UI</li> <li><code>schedule:</code> to run the jobs on a cron timed basis <a href="https://crontab.guru/#0_1_*_*_6">every week on Saturday</a></li> </ul> <p>Notice that there is no setup of ChromeDriver because the in-built Selenium Browser Manager is taking care of that step for me.</p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Recruit And interview people. Do not just fill roles https://www.eviltester.com/blog/eviltester/career-advice/recruitment/recruiting-and-interviewing-testers/ Sat, 15 Jul 2023 12:26:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/career-advice/recruitment/recruiting-and-interviewing-testers/ <p>Are you recruiting people, or are you recruiting to fulfil a role? Here are tips and notes about recruiting and interviewing.</p> <h2 id="recruiting-vs-filling-a-role">Recruiting vs filling a role</h2> <p>Too many companies seem to want to fulfil a role and bring in a candidate who ticks the right boxes in theory; instead of hiring individuals who can do most of what you want now, learn how to do the rest, and can offer you more than you thought you needed and take you to a level you never imagined.</p> <p>When you recruit to fulfil a role you might: assess against a checklist, objectively weight points against specific attributes, and when you employ, you then squeeze someone into a rigid set of responsibilities.</p> <p>When you recruit people, you gain access to their full range of experience and potential, and allow them to overspill the role and benefit your team in ways you didn&rsquo;t anticipate.</p> <h3 id="start-with-the-job-description">Start with the Job Description</h3> <p>Anyone who has written a job description before knows that it is hard. It is a painful and bureaucratic activity.</p> <p>How tempting then, to simply re-use one that you&rsquo;ve written before, changing a few words here and there, or having a standard template for your company/department/team and then fill in the blanks and expand a little.</p> <p>If you are recruiting to fulfil a role then re-using a previous description or a template is a good way to start. After all, you don&rsquo;t really want to attract someone flexible that can add value in ways you don&rsquo;t expect, you want them to conform to the role description.</p> <p>A job description I saw this morning fit this model perfectly.</p> <ul> <li>The Job Title was &ldquo;Test Manager&rdquo;, but the job description was a copy and paste from a previous role because the first line of the description read &ldquo;Test Analyst &hellip;&rdquo;.</li> </ul> <p>There was repetition of this throughout the job description. Clearly not much thought had gone into the writing of this spec, after all when fulfilling a role, you don&rsquo;t need to attract the best person, you just need someone who matches the listed criteria.</p> <ul> <li>Description mandated &ldquo;at least 4 years in&hellip;&rdquo;.</li> </ul> <p>When recruiting for a role having the ability to reject people based on years of working in a previously titled role makes it easy to evaluate. Very easy to review CVs and reject them based on objective criteria without having to consider what people learned over those years in the role.</p> <ul> <li>When asking for good communication skills, ensure that your job description communicates well. Having frequent spelling mistakes and poor grammar does not set a good standard of the communication skills that you expect. e.g. &ldquo;Excellent communication ability, analytic and report to the manager&rdquo; does not set a high standard.</li> </ul> <p>The Job description is the first, and possibly only, view that the candidate has into your organisation. If it communicates poorly then the candidate can rightly assume that you communicate poorly, and do not value good communication, or that you have incredibly low standards.</p> <p>Unfortunately I could go on and pull apart a specific job description, but there is little point. It is more important that we fix the description to recruit the best people.</p> <h3 id="fix-your-job-description">Fix your Job Description</h3> <p>If you do want to recruit people then remember that your Job Description is not simply a statement of what you want, it is also a filter. People read the description and then choose to apply, or reject it. If you don&rsquo;t put care and attention into the job spec, people who do care will reject it and you have created a filter that draws in people who care less.</p> <p>The number of years is less important than the lessons learned, the skills gained and the experience built up. If you can specify what experience you want to bring into your team and workplace then that will serve you better than years worked. This also opens up more interesting and valuable conversations in the interview. &ldquo;What did you learn&hellip;&rdquo;, &ldquo;Given your experience, how would you approach it differently&hellip;?&rdquo; rather than &ldquo;How many years in&hellip;?&rdquo;</p> <p>The Job Description is your chance to reveal your culture and values to the candidate, if they do not shine through then you run the risk of attracting someone who works better in a different organisational culture. If you communicate your culture poorly then someone who values that, will filter themselves out of the process.</p> <p>You know what gaps you have on your team, or in your department:</p> <ul> <li>skill set gaps</li> <li>experience gaps</li> <li>interpersonal communication gaps</li> <li>whatever</li> </ul> <p>In the job description, describe the problems that you think having those gaps has led to - phrasing them as solutions.</p> <ul> <li>&ldquo;You will help us automate the execution of a web application using Java to add acceptance condition validation during the build process.&rdquo;</li> </ul> <p>For a more detailed example, compare the following two statements:</p> <ul> <li>&ldquo;We are trying to move from waterfall to Agile and need people with previous experience of Agile who can work on teams to help improve the way we work.&rdquo;</li> <li>&ldquo;Experience working with waterfall and agile methodologies.&rdquo;</li> </ul> <p>If your job description simply states what you want, rather than why you want it, then the candidate has no ability to infer a context from your statement e.g. How should someone interpret the second statement &ldquo;Experience working with waterfall and agile methodologies.&rdquo;</p> <ul> <li>Do you do both?</li> <li>Are you transitioning?</li> <li>Do you actually know what these words mean?</li> <li>Do you have a hybrid process?</li> <li>How do you expect me to apply my experience?</li> <li>What is the lack of experience leading to?</li> </ul> <p>The first statement &ldquo;We are trying to &hellip;&rdquo; provides a context and implies that you need to have experienced waterfall, and agile methodologies, and know how to transition them. It also provides an indication of the challenges the role involves and is more of an attract statement rather than a filtering out statement.</p> <p>A current trend, with the advent of Chat GPT, is to use LLM tools to generate portions of the job description. I&rsquo;ve seen &ldquo;written with the help of Chat GPT&rdquo; as a badge of honour in some descriptions. Presumably, I&rsquo;m supposed to read that and think &ldquo;Wow, what a hi-tech and advanced company, using AI in the recruitment process&rdquo;. In reality, I read that and think &ldquo;Wow, what a lazy recruitment process, they can&rsquo;t even be bothered to properly craft a job description&rdquo;.</p> <h3 id="attract-the-best">Attract the Best</h3> <p>To fulfill a role, make statements that can be answered &ldquo;Y/N&rdquo; and are easy to filter CVs through. This will not attract the best people to your role because they need to see that you understand why you are recruiting and can appreciate the value that they can bring to your organisation.</p> <p>To recruit people, write your job description as a sales pitch which describes your environment, the challenges you face, and the gaps that you need help with.</p> <p>To write job descriptions like this you need to more fully understand why you are recruiting someone in the first place.</p> <p>I don&rsquo;t recruit just to meet numbers of staff, I recruit because there are specific aspects of the work that I need to improve. And to do that, I need to understand my team, teams, or department well.</p> <p>As a consultant, part of my work is helping managers understand what the gaps are, and how they can best fill those gaps. Also helping them shape the recruitment process to enable them to recruit the staff they need.</p> <p>You want to attract the best, not have them reject you because your job description fails the test.</p> <h2 id="recruitment-process">Recruitment Process</h2> <p>You have four main opportunities during the recruitment process to recruit the right person:</p> <ul> <li>filtering in or out via the job description</li> <li>reviewing the CVs and cover letter</li> <li>interview over the phone</li> <li>audition face to face</li> </ul> <h3 id="using-job-descriptions-to-funnel-in">Using Job Descriptions to Funnel In</h3> <p>You want the job description to attract people with the flexibility to excel in the job. Most job descriptions I see, filter out people who can excel because they are so poorly written that they put people off, rather than attract them in.</p> <h3 id="a-bad-process-filters-out-good-people">A bad process filters out good people</h3> <p>A bad recruitment process can filter out good people.</p> <p>Approaches that have caused me to drop out of recruitment processes:</p> <ul> <li>multiple choice coding tests</li> <li>take home coding exercises</li> <li>writing essays about my background</li> <li>creating presentations</li> </ul> <p>All of the above mean a very formal process that does not factor in the individuality of the candidate.</p> <p>I turned up for an interview and was asked to do a multiple choice coding test. Had I known this was part of the process I would have pulled out earlier. Immediately upon turning up at the company I was taken into a room, and left with a bit of paper and a pen for 30 minutes. I was finished in ten minutes and had to sit bored for twenty minutes. Then I had to wait for an additional twenty minutes as the test was marked, only to be told that I had passed &ldquo;like they knew I would&rdquo;. I wasn&rsquo;t being treated as an individual human, I was being treated as a gradeable cog in a machine. The interview process was pointlessly bureaucratic and inflexible which didn&rsquo;t create a favourable impression of the company.</p> <p>I was consulting at a company that used online multiple choice coding tests as part of their filtering process, and they would only move forward with people who gained more than 80% on the test. I paid to go through one of the tests they used, in a programming language that I didn&rsquo;t know. I achieved a mark of more than 80%. I suspect that my lack of knowledge meant that I was able to interpret ambiguity in the question in a way that someone with more experience might stumble and go &ldquo;do you mean X or Y?&rdquo; &ldquo;well, it depends on the scenario, are we doing Thing A, Thing B, or Thing C&rdquo;? I suspect the test filtered out people that were experienced, rather than filtered out inexperience.</p> <p>I dislike take home coding tests. I usually drop out of interview processes that use them. Very often the team creating them haven&rsquo;t actually done the exercise. Often the team creating them have more domain knowledge and underestimate how long the test will take. I&rsquo;ve never seen any company that uses coding tests, look at the online portfolio of a candidate, which again means that the people coming through the recruitment process are not being viewed on the individual skills and what they can uniquely bring to the company, they are being evaluated as a homogenous resource.</p> <p>Even though many of the above items are used to avoid bias in the recruitment process, I think they bias towards people who have time. These tests are harder for people to do if they have young children, or a current job that takes a lot of time, or a whole host of non-skill related responsibilities. If someone is actively looking for work and are faced with 3 or 4 hours of extra work for each application, that doesn&rsquo;t feel like the most inclusive recruitment process.</p> <p>I once pulled out of a recruitment process that required me to write an essay on my background and how well I did at school in science and maths. After more than 20 years of experience, I was being asked about Maths at school, rather than the 100+ projects I&rsquo;d created on Github. That didn&rsquo;t seem like a process that was designed to recruit experienced staff.</p> <p>I&rsquo;ve dropped out of processes that required me to create presentations and then present them in the interview. I could do a presentation. I&rsquo;ve prepared and presented hundreds over the years, at conferences and the evidence is online.</p> <p>Most of the time I drop out because I see from the process that the company is not interested in me as an individual. Ther are interested in a standarised process. The standardised process takes a lot of the work off the company and puts it all on the candidate. Most of the work on the candidate is assigned without the candidate even having spoken to someone at the company or identifying if the company is a good fit for them. I take the view that if the recruitment process is not a good fit for me, then the company has actively created the process to filter people like me out, and therefore I have to assume that I&rsquo;m not a good fit for the company and drop out.</p> <p>I don&rsquo;t think companies even track the number of people who drop out of the process when they learn what it involves.</p> <p>And companies can&rsquo;t track the number of people who never apply due to their pre-knowledge about the process.</p> <p>It is important to review recruitment processes for all the filter points that can happen and make sure that the company is aware.</p> <p>Companies often say that they &lsquo;recruit the best&rsquo;. The reality is that they &lsquo;recruit from the subset of people that comply with the recruitment demands&rsquo;.</p> <h3 id="ask-for-a-cover-letter">Ask for a Cover Letter</h3> <p>Many people do not tailor their CV to the job description. Primarily because most job descriptions are so poorly written and generic that there is no need. When you write a good job description encourage the writing of a cover letter which allows the candidate to re-use their standard CV, and write an interpretation of how they can fulfill your well-written job description.</p> <p>The process should honour the time commitments of both candidate and employer. A template driven job description values the time of the employer because it was fast to produce, but dishonours the candidate because they have to spend time interpreting what you mean. Very often the expectation from the employer is a custom written CV to match their vague requirements - again this takes a lot of time. It is faster for the candidate to write a cover letter, and this provides you with more insight into how they have interpreted your role.</p> <p>Review the CV to build a list of questions which you can use during the phone interview.</p> <h2 id="interviews">Interviews</h2> <h3 id="phone-interviews">Phone Interviews</h3> <p>Phone interviews are important because they honour the time of everyone involved.</p> <ul> <li>No travel time or cost</li> <li>Agreed time</li> <li>Agreed duration</li> <li>Expectation of preparation on both sides</li> </ul> <p>As an interviewer or interviewee, a phone interview has the advantage that you can cut it short if it is obvious that there is a mismatch between the candidate and the role.</p> <p>Use the phone interview to ask questions of the candidate about the CV, if the CV doesn&rsquo;t describe their experience very well, then ask questions to find out what they learned and how they would approach the task differently.</p> <p>Phone interviews are hard. They should be performed by someone with the requisite skill to understand and ask detailed follow-up questions based on the answers. This is particularly true if you want candidates to perform technical activities, the interviewer must know how to perform those activities well otherwise they may be misdirected by technical language.</p> <p>Ensure that the candidate asks questions to help them decide if the role or company culture is a good fit for them.</p> <p>No-one wants to waste time in a face to face interview for a candidate without applicable experience, or for a non-suitable role or culture.</p> <h3 id="audition-face-to-face-interview-over-the-phone">Audition face to face, Interview over the phone</h3> <p>The face to face interview should be an audition. If you spend all the time talking then you have wasted an opportunity to see the candidate perform the job in action. Try to recreate activities they would do as part of the job. This means avoiding:</p> <ul> <li>written questions &ldquo;to test for knowledge&rdquo; - if you require this, do this way before a face to face meeting</li> <li>whiteboard coding challenges - have them actually write code</li> <li>talking exclusively - this allows people who can &rsquo;talk the talk&rsquo; into your organisation, despite them not being able to &lsquo;walk the walk&rsquo;</li> </ul> <p>When I am recruiting for testers, either for my own teams when working as a manager, or as a hired consultant helping companies recruit good staff, I audition based on the problems we want this person to help solve or the skill gaps that we perceive we have.</p> <h2 id="interviews-as-auditions">Interviews as Auditions</h2> <h3 id="recruiting-for-exploring">Recruiting for Exploring</h3> <p>If we want to recruit a tester with experience of testing web applications, interacting with them through exploratory testing and going deep into the technicalities of the application to expose problems and identify risk then I would:</p> <ul> <li>sit them down in front of a computer where they can test the application</li> <li>restrict the testing to a small subset of the application</li> <li>provide a high-level summary of the functionality and a demo of it in use</li> <li>pair with them as they test the application</li> </ul> <p>If they are not used to explaining what they do as they test, then I would ask questions like:</p> <ul> <li>what made you do that?</li> <li>what objective do you have in mind at the moment?</li> <li>etc. to help them verbalise their thought processes</li> </ul> <p>I might even ask:</p> <ul> <li>does that look right? asking about a particular observation on screen</li> <li>etc.</li> </ul> <p>Lots of etc. because this is an open-ended process where I want to gain an insight into how they approach the testing.</p> <p>I also know that the audition setup is not how they would normally organise their work so I want to know what omissions we have:</p> <ul> <li>How would you normally communicate the testing you perform?</li> <li>Would you take notes? how would you take notes?</li> <li>Are there any additional tools you would use for this type of work? How would you use those tools? How do those tools help you? What other tools can you use to do that?</li> <li>Are you checking results in the system at all the places you would want to check them?</li> <li>etc.</li> </ul> <p>Then I would let them direct the action. If you were testing this for real:</p> <ul> <li>What else would you need to know?</li> <li>How would you approach it?</li> <li>Are there any risks you would be concerned about?</li> <li>etc.</li> </ul> <h3 id="recruiting-for-automating">Recruiting for Automating</h3> <p>If we have gaps around automating then I need to know if the candidate can:</p> <ul> <li>identify strategies and approaches</li> <li>read code</li> <li>maintain code</li> <li>improve code</li> <li>write code</li> <li>architect code</li> </ul> <p>After a demo:</p> <ul> <li>How would you automate this?</li> <li>What do you think would be hard about automating this?</li> <li>How would automating this help?</li> </ul> <p>Show some code we have written to automate this:</p> <ul> <li>How does this code work?</li> <li>What does it do?</li> <li>Does it do it well?</li> <li>How might you write it differently?</li> </ul> <p>Have some code that doesn&rsquo;t work:</p> <ul> <li>Does this work?</li> <li>Can you make it work?</li> <li>Can you now make it better?</li> </ul> <p>Can you implement some new execution paths?</p> <p>How would you structure the code to make it more maintainable and adaptable in the future?</p> <p>Given how we have written this, and the tools we have used, are there any parts of our test process that this does not support?</p> <p>Could we use this to test the system under load? Is that the best way to do this?</p> <p>etc etc.</p> <h3 id="make-it-real">Make it Real</h3> <p>I used to use MS Paint as a &rsquo;test this&rsquo; application.</p> <p>I had 2 charters:</p> <ul> <li>A &ldquo;risk identification tour&rdquo; charter</li> <li>A Scenario charter: &ldquo;Function A&rdquo; has changed - the existing automated tests have passed but it has a lot of internal dependencies - can you give it a quick once over and a quick regression of &ldquo;sub function Y&rdquo; and let us know what you find?</li> </ul> <p>But&hellip; I&rsquo;ve never worked on MS Paint. I never will. We don&rsquo;t test applications like MS Paint. So it was a non-representative exercise.</p> <p>As much as possible we want to see them in action, doing the stuff they would do in the real world.</p> <p>So I&rsquo;d rather test or automate the actuall applications that we work with and make the hands on sessions as real world as I can.</p> <p>Similarly I&rsquo;d never ask someone &ldquo;How would you test this pen?&rdquo; or &ldquo;How would you test this chair?&rdquo;.</p> <p>Because I know that my response to those questions would not be favourable - &ldquo;I&rsquo;m sorry I thought the job description was testing web applications, not pens&rdquo;. I&rsquo;d feel as though we were wasting time and that I was being patronised. I&rsquo;d react with scorn and feel my time was being wasted on &lsquo;cute&rsquo; exercises rather than a serious attempt to see if I was a good fit for the company. I&rsquo;d feel as though the company was not a good fit for me.</p> <p>This might be more challenging for management roles that are less hands-on, but you can simulate scenarios or ask how they would handle specific situations, and how they have handled them in the past.</p> <p>When you audition, you can very quickly see gaps in the experience of the candidate, and then you can decide if that gap is a blocker to hiring or something that you can work around.</p> <h3 id="experienced-evaluation">Experienced Evaluation</h3> <p>I find it much easier to evaluate candidates with an audition process, after having filtered them out through a talking process over the phone. This also avoids the need for multiple face to face interviews which often serve to prioritise company time, above candidate time.</p> <p>Throughout this process, it is essential that you have experienced people evaluate the candidate. It is too easy to talk the talk to someone that doesn&rsquo;t know how to do the role, and it is too easy to bamboozle someone in an audition if they don&rsquo;t know how to actually do the work themselves.</p> <p>All of this is about helping your candidate make the best impression on you, and demonstrate their skills as an individual and the additional value they can provide. Rather than trying to fit them into a role based on what they say they can do.</p> <h2 id="interviewing-nuances">Interviewing Nuances</h2> <p>Normally when we recruit someone it is because we:</p> <ul> <li>have stuff we want them to do</li> <li>have ideas of the ways we want them to do it</li> <li>know why we want things done that way</li> </ul> <p>But we often don&rsquo;t put that in job specifications, because the people doing early CV filtering can&rsquo;t filter on those items, because the people filtering lack the required experience necessary to draw the above out of a CV to identify the people that we might want to work with who can add value in our teams.</p> <p>I try to interview based on the person, and I will ask questions around the stuff we want them to do:</p> <ul> <li>how do they interpret what I have said?</li> <li>have they done anything that they think is similar?</li> <li>what did they learn from doing that?</li> <li>what worked? what didn&rsquo;t?</li> <li>how would they do it next time?</li> </ul> <p>There are no right answers, I have to interpret the answers based on my experience. I&rsquo;m also looking to see what else this person can do, and determine if those experiences or thought processes could help us (perhaps what they offer is more important than the capabilities we were seeking).</p> <p>A <em>role</em> based approach will not help you hire <em>people</em> who can add value beyond what you were looking for, or change what you thought you were looking for.</p> <p>During the questioning I am building a model of this person and how I think they might fit into the team and do the tasks we want done. I ask questions to build this model. I want them to ask questions of me to question my model of the work and envisioned required capabilities.</p> <p>I also have them engage in an audition, where I pair with them to:</p> <ul> <li>do they type of stuff we want them to do e.g. <ul> <li>maintain code that automates the application,</li> <li>review code to figure out how it could be better.</li> <li>test the system</li> <li>review the system to see what risks they identify</li> <li>explain the system and engage in an iterative Q&amp;A session so they can build a model of the system and explain how they might test it</li> </ul> </li> </ul> <p>Rather than questions, I have samples of work and systems and I work with them or question them to see how they deal with the work and systems.</p> <p>I might have a list of questions to prompt me during the interview to see if my model of the person covers these areas:</p> <ul> <li>how does this person stay up to date?</li> <li>what have they learned from their testing in the past?</li> <li>what have they learned from automating in the past?</li> <li>do they have absolute and rigid beliefs or approaches related to the job?</li> <li>etc.</li> </ul> <p>I generally don&rsquo;t ask these as questions to them, these are questions to me, to allow me to check my model of the person.</p> <p>I adjust the audition around the person based on the job, first by having them explore my model of the job and discussing how their experiences map to the model, and then a hands on practical audition for aspects of the role.</p> <h2 id="sometimes-the-process-is-bad">Sometimes the process is bad</h2> <p>Sometimes we don&rsquo;t control the process. Other people decide on the recruitment process and we might be forced to comply or take part in it.</p> <p>This just means that we have to approach the process with as much flexibility as possible and help the person shine through.</p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Testing The Triangle Application https://www.eviltester.com/blog/eviltester/apps-i-wrote/the-triangle-application/ Sat, 15 Jul 2023 08:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/apps-i-wrote/the-triangle-application/ <p>The Triangle Problem is a &lsquo;classic&rsquo; from Software Testing, described in many books and training courses. This post describes the problem, links to some applications you can use to practice on and describes some nuances around the testing.</p> <h2 id="what-is-the-triangle-problem">What is The Triangle Problem?</h2> <p>The application context surrounding the Triangle Problem is usually:</p> <blockquote> <p>Given an application which takes three inputs, each representing the length of one side of a triangle, the application will respond with a message identifying if the inputs are: invalid, or represent a triangle, if they represent a triangle then the application will respond with the type of triangle (equilateral, isosceles, scalene).</p> </blockquote> <p>The &lsquo;problem&rsquo; is usually presented as a test design problem:</p> <ul> <li>How would you test this?</li> <li>What inputs would you use to test this?</li> <li>etc.</li> </ul> <h2 id="implementations-of-the-triangle-application">Implementations of the Triangle Application</h2> <p>I have written an implementation of the <a href="https://testpages.herokuapp.com/styled/apps/triangle/triangle001.html">Triangle Application</a> as part of my <a href="https://www.eviltester.com/page/tools/testpages/">Test Pages</a> application.</p> <ul> <li><a href="https://testpages.herokuapp.com/styled/apps/triangle/triangle001.html">Test Pages Triangle Application</a></li> </ul> <p>During my research I found an implementation by <a href="https://www.mathwarehouse.com">MathWarehouse.com</a> the Triangle Calculator below:</p> <ul> <li><a href="https://www.mathwarehouse.com/triangle-calculator/online.php">Triangle Calculator</a></li> </ul> <h2 id="triangle-problem-exercises">Triangle Problem Exercises</h2> <h3 id="traditional">Traditional:</h3> <ul> <li>In advance of testing, create a set of inputs with expected results, try them in the application, review coverage</li> <li>Work with others, repeat exercise above, compare inputs and coverage</li> <li>Create a &lsquo;minimum&rsquo; input set for your testing.</li> </ul> <h3 id="using-either-of-the-implementations">Using either of the implementations:</h3> <ul> <li>The input design ignores the GUI implementation, what else do you need to do to test the GUI?</li> <li>Test the drawing functionality, does that require different test input?</li> <li>Can you spot any hidden functionality? If so, test it.</li> </ul> <h3 id="discuss">Discuss:</h3> <ul> <li>When would creating a &lsquo;minimum&rsquo; input set be helpful?</li> </ul> <h3 id="tool-support-working-with-test-pages-implementation">Tool Support working with Test Pages Implementation:</h3> <ul> <li>Are there any lightweight adhoc tools or approaches for automating that could help you as you perform exploratory testing?</li> </ul> <h3 id="automating-test-pages-implementation">Automating Test Pages implementation:</h3> <ul> <li>Review the implementation from the perspective of &ldquo;Is this automatable?&rdquo;</li> <li>How many different ways could you automate this?</li> <li>Would any changes to the implementation make automating easier?</li> <li>Try automating the implementation.</li> </ul> <h3 id="code-review">Code Review:</h3> <ul> <li>Review the code for the Test Pages Implementation, does that help or hinder your testing?</li> </ul> <h3 id="using-the-test-pages-implementations">Using the Test Pages implementations:</h3> <ul> <li>Unit test the underlying JavaScript implementation, compare with your GUI approach and original test design.</li> <li>Automate the Unit Testing from the console.</li> </ul> <p>Make up your own exercises to push your approach to testing this simple example as far as you can.</p> <h2 id="books-and-research-resources">Books and Research Resources</h2> <p>The Triangle Problem is described and referenced in the resources and books that follow.</p> <h3 id="books">Books:</h3> <ul> <li>The Art of Software Testing - Glenford Myers</li> <li>Software Testing a Craftsman&rsquo;s Approach - Paul C. Jorgensen</li> <li>Testing Object-oriented Systems - Robert Binder</li> <li>Test Driven Development By Example - Kent Beck</li> </ul> <h3 id="additional-resources">Additional Resources:</h3> <ul> <li>Exercise: Analyzing the Triangle Problem by Ross Collard from the Third Annual Workshop on the Teaching of Software Testing 2004 <ul> <li>testingeducation.org used to have easy to find links to this material and details of the conference but the site seems to have invalidated the content.</li> <li>there are various sites which have archived this pdf</li> <li><a href="https://www.yumpu.com/en/document/view/13190450/exercise-analyzing-the-triangle-problem-testing-education">https://www.yumpu.com/en/document/view/13190450/exercise-analyzing-the-triangle-problem-testing-education</a></li> <li><a href="https://pdf4pro.com/view/exercise-analyzing-the-triangle-problem-testing-9fc7.html">https://pdf4pro.com/view/exercise-analyzing-the-triangle-problem-testing-9fc7.html</a></li> </ul> </li> </ul> <p>Expanded content on The Triangle Problem was <a href="https://www.patreon.com/eviltester/posts?filters%5Btag%5D=triangle%20problem">released to Patreon Supporters back in 2022</a></p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; On Modelling https://www.eviltester.com/blog/eviltester/modelling/on-modelling/ Mon, 10 Jul 2023 15:30:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/modelling/on-modelling/ <p>TLDR: Every time we test something we are testing from models. Modelling is a key skill for Software Testers. Errors cannot be identified without a model to compare them. Quality Control cannot be conducted without a model.</p> <h2 id="modelling">Modelling</h2> <blockquote> <p><em>&ldquo;A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness&hellip;&rdquo; A. Korzybski, Science &amp; Sanity, 4th Ed. 1958, pp. 58-60 (quoted from: R Bandler &amp; J Grinder, Patterns of the hypnotic Techniques of Milton H. Erickson, M.D. Vol 1, 1975, pp 181)</em></p> </blockquote> <p>Human beings model the world. We learn by constructing models and then subjecting those models to tests in order to determine their validity. Human beings do this all the time. By the time a human reaches adulthood it has constructed so many models that the person is probably unaware of all those models, and may even be unaware of the modelling process.</p> <p>When we are faced with something new which we don&rsquo;t understand, a new fangled tap in the washroom for example, we will use the models that we have already developed which are similar to that situation; all our tap models. If they fail to make the tap work then we will use other models associated with the washroom. Having encountered hand driers with proximity sensors we may wave our hands around to get the tap working. If we have been on a train or a plane then we may have encountered taps that work through foot pedals and we would start to look for those. We have models for turning things on outside the washroom using buttons or levers and we would start using those.</p> <p>We use models and strategies all the time. We are experts at constructing models and strategies to apply to those models. We interact with the world through those models.</p> <p>The software development process constructs software that works more often than it fails. This is in spite of us not doing what we are told is best practise, or even what we believe we should do; we don&rsquo;t spend enough time on requirements, we don&rsquo;t stabilise requirements, we don&rsquo;t design, we don&rsquo;t unit test, we don&rsquo;t document, we don&rsquo;t review. We often get away with it because modelling is a natural talent and software development is a process of modelling.</p> <p>Unfortunately we also learn from experience, and if our experience of success includes not doing all these best practise processes then that leads us to not do these processes again.</p> <p>We have to understand what we do and what each step is for, in order to ascertain if we can miss them out in the situation we are in.</p> <p>Modelling explicitly, and understanding our models, allows us to be pragmatic.</p> <h2 id="modelling-software-development">Modelling Software Development</h2> <p>Software development tries to create a product. It does this by engaging in a number of processes (requirements, design, coding, etc.). Each of these processes will create a model that will either be the final product or an input to a follow on process.</p> <p>There are constraints on the construction of a product; we need it in X days, it must only cost Y thousand, and you only have Z staff members. The skill of developing software is in the effective application of the strategies that have been learned bearing in mind the constraints involved.</p> <p>Modelling is a fundamental activity of the software development process.</p> <p>To take the main processes from a minimal software life cycle:</p> <ul> <li>Requirements <ul> <li>Requirements are modelled, possibly as text.</li> </ul> </li> <li>Design <ul> <li>The popular UML provides a range of diagrams: Class, Object, Component, Deployment, Use Case, Sequence, collaboration, Statechart, Activity</li> </ul> </li> <li>Coding <ul> <li>The program is modelled in Code. We have the choice of language to model the system in be it C++, Smalltalk or assembly. The code is a model. When we execute the system we have to have special programs to map that executable system back into the code model. e.g. a source code debugger. (It is possible to formalise the design models above so that they are equivalent to a code model.)</li> </ul> </li> <li>Testing <ul> <li>Testing can use many of the development models and will apply strategies such as: loop once, loop twice, cover every statement, cover every predicate condition, cover every exception.</li> </ul> </li> </ul> <p>Each of the models produced in the development process is a refinement of a previous model, even if the previous model was never formally documented. A requirement is a refinement of the dreams and aims associated with the picture in the specifier of the requirement&rsquo;s head.</p> <h2 id="modelling-for-process-improvement">Modelling For Process Improvement</h2> <p>The simplest way to improve a process is to analyse the errors that it lets slip through.</p> <p>For every error not found by one of the previous quality control processes, ask, is there a strategy that could have been applied to one of the existing models that would have created a test case that could have identified the error.</p> <p>If the answer is yes then it may well have slipped through because timescales or staffing levels forced your hand and you simply didn&rsquo;t have the time to apply that strategy to that model, or the risk of not applying it was deemed low.</p> <p>If the answer is no then we have to identify a model and a strategy that could have found it.</p> <p>In both cases assess the cost and time impact of adopting that model and strategy. The development process is one of trade offs and compromises.</p> <h2 id="modelling-and-testing">Modelling and Testing</h2> <p>In order to test a model we have to have some way of recognising the success or failure of a test, our testing has a goal, it has a reason for existing.</p> <p>That reason is inherent in the model from which the testing is derived.</p> <p>This means that it is difficult to use a model to test itself.</p> <p>In software development this is typically not a problem, we rarely use the source code as the only model when testing the source code. We typically derive tests from a design model, a requirements model or a specific testing model, or even a model which may be derived from the source code, essentially any model which has the level of detail required for our testing.</p> <p>Modelling is as fundamental an activity of testing as it is of development.</p> <p>Test Strategies are applied to models. This is for the purposes of test derivation, derivation and execution coverage measurement, domain analysis, risk analysis, the list includes almost every task that testers do.</p> <p>Strategies typically evolve and are identified by thinking about errors that slipped through and identifying a strategy that could have found them.</p> <h2 id="testing-and-modelling-using-exploration">Testing and Modelling using Exploration</h2> <p><em>Testing is exploration. In a mature testing organisation the expedition is well planned, and staffed with seasoned explorers. The planning will be done around a number of maps of the territory to be explored. Some maps will show different levels of detail - to show all the detail on one map would confuse the issue, so one map will identify areas of population, one will provide information of season rainfall statistics etc. Maps are very important. The explorer plans different routes through the maps to match the aims of the expedition, perhaps they are trying to unearth hidden temples and consequently will pick routes which take them through areas which are sparsely populated now, but in the past were densely populated. Effective exploration requires an understanding of the terrain to be explored.</em></p> <p>Errors cannot be found without a model. Quality Control cannot be conducted without a model.</p> <p>I have heard it said that &ldquo;some testers never model&rdquo; and &ldquo;reviews are not conducted against a model&rdquo;. These statements are false. In the absence of a defined and identifiable model, there will be an informal model, a model of understanding in the tester&rsquo;s head.</p> <p>A model is our understanding and knowledge of the system. The level of testing that can be done with no understanding and no knowledge of the system is zero.</p> <p>Try this. Take a program that you don&rsquo;t know what it does. Make sure that the program presents all its information in a way that you cannot understand. If you don&rsquo;t know Japanese then test a Japanese program. If the information presented to you is obscure enough then you will find it impossible to build a model of it and then you have no way to assess the correctness of any action. Remember that if you even understand the name of the program or its main purpose then that is information that you will have assimilated into a model and will use during testing.</p> <p><em>Note: you might make the application crash, you probably know how to recognise an application crashing, even if you don&rsquo;t speak the language. You spot problems like a crash, because you have model of crashing applications which are independent of a language model.</em></p> <p>Reviews cannot be conducted without a model of whatever the thing being reviewed is supposed to represent. Review models are different from testing models.</p> <p>A review will be conducted against a number of models:</p> <ul> <li>The model of a well-formed document. (does it have a title page? Are the pages numbered?)</li> <li>The syntax of the actual text</li> <li>The semantic model in the reviewers&rsquo; head of the items to be presented which they have a vested interest in.</li> </ul> <p>There are at least as many informal models as there are people.</p> <p>Modelling is a fundamental task in testing.</p> <p>Quality Control is essentially the checking of a model against an implementation of that model.</p> <p>A test is a specific situation with a predefined set of things to check against the model. The differences are errors, either in the model or the implementation.</p> <h2 id="test-conditions-as-modelling">Test Conditions as Modelling</h2> <p>Test Conditions are statements of compliance which testing will demonstrate to be either true or false. These are (in effect) test requirements.</p> <p>Conditions serve different purposes. Some conditions will act as the audit reason for a particular test case e.g. The user must be able to create a flight. The tester will create a test which creates a flight, obviously there are more attributes to this case than this - what type of flight, what type of user, fully booked, partially booked, etc. These attributes are other types of conditions.</p> <p>Some conditions are used to define a test&rsquo;s attributes or preconditions. E.g. create flight of type local, create flight of type international.</p> <p>Or are they&hellip;.</p> <p>This may be modelling that has not gone far enough.</p> <p>The initial condition &lsquo;create a flight&rsquo; is valid. When we only have test conditions as our modelling tool then we have to represent this as a condition. It is also a program function - create flight, or an object method, or an entity event, or a business process. Consequently we should really have a model and a derivation strategy that says &ldquo;there must be at least one test for each entity event&rdquo; or &ldquo;there must be at least one test for each object method&rdquo;. In this case it is obvious that &ldquo;one test&rdquo; will not cover the condition but with a rich model, with object or entity models we have a list of properties or attributes, these will have scoping variants (i.e. attribute flightType - international, local).</p> <p>Basically, we use these context rich models to give us the combination information that we require to construct test cases. Without this approach we will never know if we have a valid or complete set of condition combinations.</p> <p>Hierarchical models are appropriate for test grouping i.e. tests related to business processes, program modules, program functions etc. There is no reason why a test cannot be in more than one test grouping.</p> <p>Hierarchical models are appropriate in derivation for hierarchical structures (it is possible to list entities attributes and events as hierarchical structures but this hides valid combination options and is a mix of ELH and ER, we should really have a relationship section on the model)</p> <p>e.g.</p> <ul> <li>Entity: Flight</li> <li>Attribute: Type</li> <li>Attribute: Start Airport</li> <li>Event: Create</li> <li>Event: Takeoff</li> <li>Event: Land</li> <li>Event: Delete</li> </ul> <p>Models for test derivation should be rich. This allows a derivation strategy to be created which can be used to gauge the completeness of the test products and the validity of the test products.</p> <p>With rich models, Test conditions become requirements which are used to check the completeness of the test derivation approach rather than the audit reason - unless of course there is no way to make the construction of test cases automatic with the implicit cross referencing of test conditions i.e. we don&rsquo;t have to state &lsquo;Create a flight&rsquo; as a test condition because there is an entity event on flight called create which we know we have to be able to test and it will apply to a variety of attributes on that entity. Without this rich modelling, and without an implicit (or strategy driven) approach to testing, a vast number of test conditions have to be created and maintained and no guarantee of combination thoroughness can be achieved.</p> <h2 id="error-guessing">Error Guessing</h2> <p>Error Guessing is described in &lsquo;Testing Computer Software&rsquo; by Cem Kaner [1]:</p> <blockquote> <p>&ldquo;For reasons that you can&rsquo;t logically describe, you may suspect that a certain class of tests will crash the program. Trust your judgment and include the test.&rdquo;</p> </blockquote> <p>This quote suggests to me that there is an informal model in the tester&rsquo;s head and that the subconscious is applying a strategy to the model which the tester is unaware of. The tester is only aware of the subconscious flagging the results of that check to the conscious as a nagging doubt.</p> <p>If you do engage in error guessing then you should be aware that:</p> <ul> <li>you have a model and applicable strategy in your head that you are not using on the project or possibly even aware of.</li> <li>if your strategy does work then you should try to quantify it so that you can use it consistently.</li> <li>If it doesn&rsquo;t work then you should possibly change the model and strategy in your head.</li> </ul> <p>[1] Testing Computer Software, Cem Kaner, Jack Falk, Hung Quoc Nguyen, 2nd Edition 1993, International Thompson Computer Press</p> <hr> <p>The above content was a slightly condensed version of five blog posts from 2002.</p> <p>You might also be interested in the following content:</p> <ul> <li><a href="https://www.eviltester.com/post/fundamentals/a-meta-model/">What is Software Testing? A Meta Model</a></li> <li><a href="https://www.eviltester.com/show/019-test-entitites-cases-scripts/">The Test Cases and Test Scripts Episode</a></li> </ul>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; June 2023 EvilTester.com and Patreon Content Summary https://www.eviltester.com/blog/eviltester/summaries/2023-06-june-2023-summary/ Sun, 09 Jul 2023 10:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/summaries/2023-06-june-2023-summary/ <p>Content Summary for June 2023 EvilTester.com and Patreon with some recommended podcasts.</p> <p><a href="https://www.patreon.com/posts/24389995">All monthly summaries of Patreon and blog content</a></p> <h2 id="overview">Overview</h2> <p>I managed to create two podcast episodes in June. I was hoping to create an episode every two weeks but it might well only be once a month. The last two episodes took quite a lot of research and thinking about to condense quite a lot of information into a short space of time and try to keep it coherent.</p> <ul> <li><a href="https://eviltester.com/show/019-test-entitites-cases-scripts">The Test Cases and Test Scripts Episode</a></li> <li><a href="https://eviltester.com/show/020-test-automation-pyramid">The Test Automation Pyramid Episode</a></li> </ul> <p>I don&rsquo;t listen to a lot of podcasts because they often take too much time. I find that I can&rsquo;t listen to a podcast in the background and take in the content so I only listen to them when I&rsquo;m out for a walk. And then when I get home, I have to re-listen to chunks in order to make notes because so many podcasts don&rsquo;t create very thorough show notes.</p> <p>To help me out, I&rsquo;ve been experimenting with Open AI models for automated transcriptions and have created a personal tool which downloads podcasts and transcribes them locally. I&rsquo;ll write more about that next month as it is one of this month&rsquo;s experiments. But the github repo can be found here as I work on it. <a href="https://github.com/eviltester/podcast-transcriber">github.com/eviltester/podcast-transcriber</a>.</p> <p>I&rsquo;m gradually adding testing and development podcasts to my auto transcribed list of podcasts and reading through the transcripts. I find it easier to let the tool work in the background for a day, and then spend an hour or so reading the transcripts than listening to 20 hours of podcasts.</p> <p>Even though the tool is pretty crude at the moment it might prove useful to speed readers like myself or people who experience hearing loss and find the audio podcast format inaccessible.</p> <p>The main podcasts I&rsquo;ve been listening to recently are:</p> <ul> <li><a href="https://copywriterspodcast.com/">copywriterspodcast.com</a></li> <li><a href="http://www.hilaritaspress.com/podcasts/">hilaritaspress.com/podcasts</a></li> </ul> <p>The Copywriters Podcast is an incredibly informative podcast on the people and techniques behind copywriting. Some of the interview shows are useful but I primarily prefer the shows where the main host David Garfinkel is covering either his experience or deep diving into the work of a classic copywriter.</p> <p>The Hilaritus Press Podcast is an interview based format with people either discussing Robert Anton Wilson&rsquo;s work or people who&rsquo;s work overlaps with Wilson&rsquo;s work in some way. The obvious episodes with an overlap to testing were the <a href="http://www.hilaritaspress.com/podcasts/claude-shannon-hilaritas-press-podcast-episode-15/">Claude Shannon Episode</a> and the <a href="http://www.hilaritaspress.com/podcasts/andrew-mcluhan-on-marshall-mcluhan-hilaritas-press-podcast-episode-14/">Marshall Macluhan Episode</a></p> <h2 id="blog-posts-for-june--2023">Blog Posts For June 2023</h2> <ul> <li><a href="https://eviltester.com/show/019-test-entitites-cases-scripts">Episode 019 - The Test Cases and Test Scripts Episode - The Evil Tester Show</a></li> </ul> <p>Test Cases and Test Scripts are Test Entities, are they useful? Could we just use Test Conditions?</p> <hr> <ul> <li><a href="https://eviltester.com/blog/eviltester/summaries/2023-0345-march-april-may-2023-summary">March, April, May 2023 EvilTester.com and Patreon Content Summary</a></li> </ul> <p>Content created and collated for EvilTester.com, Patreon and Social for March, April and May 2023</p> <hr> <ul> <li><a href="https://eviltester.com/show/020-test-automation-pyramid">Episode 020 - The Test Automation Pyramid Episode - The Evil Tester Show</a></li> </ul> <p>What is the Test Automation Pyramid? Is it for Testing or Automating?</p> <hr> <h2 id="patreon-posts-for-june--2023">Patreon Posts For June 2023</h2> <ul> <li><a href="https://www.patreon.com/posts/arguing-against-85314636">Arguing against automating</a></li> </ul> <p>Arguing against automating prior to having the ability to explore effectively.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/shift-left-or-85154848">Shift Left or Normal Development</a></li> </ul> <p>Is shift left just normal effective Agile Development?</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/web-seo-link-85096708">Web SEO Link checking Tools</a></li> </ul> <p>Web and SEO checking tools.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/kotlin-docs-84863912">Kotlin Docs</a></li> </ul> <p>Kotlin has very good documentation, and the examples are editable and runnable in the browser.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/020-test-pyramid-84750365">020 The Test Pyramid episode Full Show Notes and Transcript</a></li> </ul> <p>Full show notes and transcript for Test Pyramid podcast episode</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/podcast-test-84716566">Podcast - Test Pyramids - Audio Version</a></li> </ul> <p>The audio version of episode 20 of The Evil Tester Show.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/podcast-test-84716453">Podcast - Test Pyramid - Video Version</a></li> </ul> <p>The video version of episode 20 of The Evil Tester Show.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/pyramid-research-84669788">Pyramid Research</a></li> </ul> <p>Notes on additional research when prepping for my next podcast episode.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/prototyping-in-84621319">Prototyping in the dom</a></li> </ul> <p>Adding some data into the DOM makes it easy to manipulate from the console.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/skip-level-84519335">skip level meetings</a></li> </ul> <p>What and why is a skip level meeting?</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/next-podcast-to-84274813">Next podcast notes to self</a></li> </ul> <p>I&rsquo;m currently making notes for my next podcast. Here I cover the order I build the outline.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/testing-like-day-84167279">Testing like a day in court</a></li> </ul> <p>Notes on the Black Belt Barrister with parallels to Software Testing</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/reviewing-as-84116624">Reviewing as a contextual testing approach</a></li> </ul> <p>Is reviewing a valid approach to gaining or reducing confidence?</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-may-84051479">Summary for May 2023</a></li> </ul> <p>This is a list of content I posted in May 2023. Back to Index of all monthly summaries Blog Posts For May 2023 Episode 018 - The Manual Test</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-2023-84051303">Summary for April 2023</a></li> </ul> <p>This is a list of content I posted in April 2023.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-2023-84051256">Summary for March 2023</a></li> </ul> <p>This is a list of content I posted in March 2023.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/may-2023-pdfs-84051110">May 2023 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon and blog content for May 2023 is attached to this post.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/april-2023-pdfs-84051080">April 2023 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon content for April 2023 is attached to this post.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/march-2023-pdfs-84050993">March 2023 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon and blog content for March 2023 is attached to this post.</p> <hr> <p><em>You can gain access to the content on Patreon for only $1 a month at <a href="https://patreon.com/eviltester">patreon.com/eviltester</a>. Supporters gain access to downloadable PDFs of the content and micro courses e.g. <a href="https://www.eviltester.com/page/onlinetraining/howitestweb-patreon/">How I Test - Web App</a>, <a href="https://www.eviltester.com/page/onlinetraining/testabilityaudit/">Test Ability Audit</a> and <a href="https://www.eviltester.com/page/patreon/#microcourses">more</a></em></p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Episode 020 - The Test Automation Pyramid Episode - The Evil Tester Show https://www.eviltester.com/show/020-test-automation-pyramid/ Sun, 18 Jun 2023 07:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/show/020-test-automation-pyramid/ <p>This episode covers the Test Automation Pyramid, created by Mike Cohn in 2008-2009 in the book &ldquo;Succeeding With Agile&rdquo;. We will go beyond the diagram and look at the model that supports it. Then deep dive into the model to explore it&rsquo;s meaning in relation to Automated Execution Coverage, not Testing.</p> <p>This was released to <a href="https://patreon.com/eviltester">Patreon supporters</a> early, with full transcript and no ads.</p> <h2 id="introduction">Introduction</h2> <p>Subtitle: The Test Automation Pyramid: Demystifying a Simple Model</p> <p>I want to talk about the test automation pyramid, or as some call it, the test pyramid. We are going to dive into its origins, the misconceptions surrounding it, and how we can interpret it in today&rsquo;s tech landscape.</p> <p>The test automation pyramid was first introduced in the book &ldquo;Succeeding With Agile&rdquo; by Mike Cohn back in 2008-2009. During that time, automation tools like Selenium were still in their early stages, and automating GUIs was challenging. The pyramid model was a reflection of the development landscape at the time, where there were process issues, outsourced testing, and proprietary automation tools.</p> <p>However, over the years, the model has been misinterpreted and misrepresented in various ways. Different variations have emerged, using terms like end-to-end tests, integration tests, manual tests, and acceptance tests: all deviating from Cohn&rsquo;s original model. This has led to confusion and disagreements about the meaning of the test automation pyramid.</p> <p>To make matters worse, some people have focused on the testing aspect of the model, rather than the automation part. This interpretation has caused unnecessary debates about the role of testers and the value of exploratory testing. But if we shift our perspective and view the model as an &ldquo;automated execution using Xunit coverage model,&rdquo; we can more easily understand the model.</p> <p>The essence of the test automation pyramid lies in achieving coverage at different levels of an application. The model suggests targeting the most appropriate level for automation, rather than trying to automate everything at the UI level. By doing so, we can avoid redundancy and duplication, reduce maintenance efforts, and speed up the automated execution process.</p> <p>Each application will require a unique approach to achieve the desired coverage. The focus should be on understanding the architectural layers, interface points, and integration risks within your specific application.</p> <p>View the test automation pyramid as a set of heuristics for achieving coverage through automated execution. Rather than getting caught up in the misinterpretations and debates, create your own model for your applications and processes.</p> <h2 id="summary">Summary</h2> <ul> <li>The model was created by Mike Cohn in 2008-2009 in the book &ldquo;Succeeding With Agile.&rdquo;</li> <li>The original model focused on UI, service level, and unit level automation.</li> <li>Over the years, different interpretations and variations of the model have emerged.</li> <li>The term &ldquo;service level&rdquo; in the model has led to ambiguity and different interpretations.</li> <li>The diagram in the model is a simplified representation of a deeper underlying model.</li> <li>The focus should be on achieving coverage at the most appropriate level in the system.</li> <li>The model addresses the importance of avoiding duplication and redundancy in automated coverage.</li> <li>The process and team structure can impact the effectiveness of the model.</li> <li>The model can be reframed as an automated execution coverage pyramid.</li> </ul> <h2 id="related-links">Related Links</h2> <ul> <li>Simon Stewart <a href="https://www.youtube.com/shorts/GNO6iHlYxzg">https://www.youtube.com/shorts/GNO6iHlYxzg</a></li> </ul> <p>James Bach presented a &ldquo;round earth test strategy&rdquo;</p> <ul> <li><a href="https://www.satisfice.com/blog/archives/4947">https://www.satisfice.com/blog/archives/4947</a> which deals with the &ldquo;Test Automation Pyramid&rdquo; from the perspective of &ldquo;Test&rdquo; rather than &ldquo;Automation&rdquo;.</li> <li>There is a video of this presentation here <a href="https://www.youtube.com/watch?v=gy93j7LE6sA">https://www.youtube.com/watch?v=gy93j7LE6sA</a></li> </ul> <h2 id="discussion-questions">Discussion Questions</h2> <ol> <li>What is the main takeaway from this episode on the test automation pyramid?</li> <li>What is important about the term &ldquo;unit process&rdquo; in relation to unit testing?</li> <li>Compare your opinion to the speaker&rsquo;s on using the test automation pyramid model to describe test automation strategies?</li> <li>Why is understanding the underlying model important when looking at diagrams?</li> <li>What are some possible gaps between the UI, service, and unit automating?</li> <li>Why is there a confusion surrounding the scope of a unit?</li> <li>What is the difference between automating through API and testing the service layer made up of classes?</li> <li>How can tooling and architecture become different when using API automation compared to UI automation?</li> <li>What process and communication challenges can lead to problems with automated execution?</li> <li>How can teams balance UI level testing with unit level testing to achieve better coverage?</li> <li>Has your understanding of the test automation pyramid changed after listening to this episode?</li> <li>What are some advantages and disadvantages of using the test automation pyramid as a model for test strategies?</li> <li>What do you think are some common misconceptions about the test automation pyramid?</li> <li>How can teams effectively split up responsibility for achieving automated test coverage between different roles or teams?</li> <li>How can teams ensure that their processes support effective test automation coverage and execution?</li> <li>What are some alternative models or heuristics that can be used to design effective test automation strategies?</li> <li>How can teams identify gaps and overlapping components in their test automation strategies?</li> <li>What are some good practices for achieving effective unit testing with Xunit frameworks?</li> <li>How do you prioritize early, fast, and smaller points in automated testing while still achieving comprehensive coverage?</li> <li>What steps can teams take to ensure proper communication and collaboration among team members when designing and executing test automation strategies?</li> </ol>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; March, April, May 2023 EvilTester.com and Patreon Content Summary https://www.eviltester.com/blog/eviltester/summaries/2023-0345-march-april-may-2023-summary/ Sun, 04 Jun 2023 10:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/summaries/2023-0345-march-april-may-2023-summary/ <p>Content Summary for March, April, May 2023 EvilTester.com and Patreon.</p> <p><a href="https://www.patreon.com/posts/24389995">All monthly summaries of Patreon and blog content</a></p> <h2 id="overview">Overview</h2> <p>I didn&rsquo;t manage to get around to uploading the summaries for March, April and May until now so I&rsquo;ve combined them all together in one post.</p> <p>A few things happened over the last few months which have taken my attention.</p> <p>Twitter changed their API access and pricing, and that completely broke my Twitter Client <a href="https://chatterscan.com">Chatterscan.com</a>. The API costs make it completely non-viable, which is a pity as it really helped me read Twitter so now I barely use the platform.</p> <p>The only functionality that I could salvage was the &lsquo;favourites&rsquo; view. So now this is a manual set of favourites which can be used as a jumping off point for brand monitoring or specific searches on sites like LinkedIn, Reddit or Google News.</p> <p>If more social media sites had APIs then I would extend the app to use them. I was just about to start experimenting with the Reddit API but then Reddit announced they were introducing API pricing.</p> <p>All of this means that social media sites will find that people start scraping the content and using unofficial APIs and their extended eco-system starts to drop.</p> <p>I investigated the User Management system <a href="https://userspice.com/">userspice</a> the documentation isn&rsquo;t great, but it is aimed at people with programing skills so I just ended up reading the code when I was stuck. But now my Patreon Bonus site <a href="https://testerhq.com">TesterHQ.com</a> has an actual membership section so I can, in the future, extend access to the content to people beyond patreon. And I have a basic escrow LMS, so when the sites that I have hosted my courses on eventually go bust, I may be able to migrate people on to my userspice powered LMS.</p> <p>This also means that if any companies are interested in getting access to the content, e.g. if a team of 10+ testers wanted to access my <a href="https://www.eviltester.com/page/onlinetraining/eviltestertalks/">Conference Talk Archive</a>, or any of the <a href="https://www.testerhq.com/member2/index.php?preview=true">Patreon Exclusive Training Courses</a> I now have a mechanism for doing that if they <a href="https://www.eviltester.com/page/contact/">contact me</a>.</p> <p>I&rsquo;ve also restarted my Podcast and although the schedule will vary I think I can probably do an episode every two weeks.</p> <p>And if you are interested in any of the <a href="https://www.testerhq.com/member2/index.php?preview=true">courses or ebooks on TesterHQ</a> or content below, then individuals can sign up to Patreon for as little as $1 a month and get immediate access.</p> <h2 id="patreon-posts-for-march--2023">Patreon Posts For March 2023</h2> <ul> <li><a href="https://www.patreon.com/posts/keep-it-simple-80778193">Keep it simple</a></li> </ul> <p>I realised that I was overthinking my ftp testing.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/testing-ftp-80675453">Testing FTP</a></li> </ul> <p>&lsquo;It&rsquo;s just a quick hack to get something working, we&rsquo;ll sort it out later&rsquo; I imagine that this was what was said when the script I&rsquo;m curr</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/role-and-80491223">Role Specialisation and Extinction</a></li> </ul> <p>&lsquo;Operating Manual For Spaceship Earth&rsquo; reminds me that over-specialisation, or the perception thereof, can lead to extinction.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/on-three-amigos-80443437">On Three Amigos</a></li> </ul> <p>What actually is 3 Amigos?</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/impact-costs-80346901">Impact Costs</a></li> </ul> <p>Many companies at the moment are getting rid of people and cutting costs. A focus on cost is almost certainly going to obscure value.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/challenge-of-out-80302094">The challenge of standing out as a human</a></li> </ul> <p>The challenge of computer generated content.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/cyoa-mapping-80049564">CYOA Mapping</a></li> </ul> <p>One of the hard things about modelling while testing is that you don&rsquo;t know the scope or form of the system when you start.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/docker-desktop-79790038">Docker Desktop Alternative</a></li> </ul> <p>Purely open source and free alternative tooling.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/chasing-error-79697035">Chasing error messages</a></li> </ul> <p>This last week I&rsquo;ve been chasing error messages.Some were helpful.Some led me down rabbit holes that were completely unhelpful.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/twitter-api-plan-79644573">Twitter API plan does not include twitter.com</a></li> </ul> <p>Twitter is broken&hellip;</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-2023-79466797">Summary For February 2023</a></li> </ul> <p>This is a list of content I posted in February 2023.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/february-2023-79466712">February 2023 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon and blog content for February 2023 is attached to this post.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/january-2023-79466561">January 2023 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon and blog content for January 2023 is attached to this post.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-2023-79466506">Summary For January 2023</a></li> </ul> <p>This is a list of content I posted in January 2023.Back to Index of all monthly summariesBlog Posts For January 2023What is Software Testing</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/integration-79406036">Integration Testing</a></li> </ul> <p>Trying to resolve differences between Automated Unit test behaviour in different environments.</p> <hr> <h2 id="patreon-posts-for-april--2023">Patreon Posts For April 2023</h2> <ul> <li><a href="https://www.patreon.com/posts/going-back-to-82330520">Going back to basics</a></li> </ul> <p>Going back to learn the basics of Unix CLI usage.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/start-simple-82193403">Start Simple</a></li> </ul> <p>GO TO is a journalistic history of programming languages.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/could-should-82143608">Could, Should, Would Automate</a></li> </ul> <p>Do testers become bad programmers?</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/twitter-api-put-81988782">Twitter API Pricing Put an End to Chatterscan.com</a></li> </ul> <p>The end of my twitter api application.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/risks-and-issues-81848980">Risks and Issues</a></li> </ul> <p>Many testing skils have a direct implementation in management.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/test-entities-81701527">Test Entities - cases, scripts, conditions</a></li> </ul> <p>The most common Testing Entities that I&rsquo;ve encountered over the years.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/checking-use-81515275">Checking language use</a></li> </ul> <p>I was thinking about use of language describing concepts.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/technology-study-81137613">Technology Study Template</a></li> </ul> <p>I was thinking through what I need to track to help me learn a technology. Particularly when trying to learn multiple technologies at once.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/more-on-sdet-vs-81088981">More on SDET vs Programmer</a></li> </ul> <p>I was thinking about the difference between programming knowledge required for automating vs that required for programming.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/backups-81041721">Backups</a></li> </ul> <p>Nas and backup options</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/sdet-vs-80985416">SDET vs Programmer</a></li> </ul> <p>I received a question via email from someone who became an SDET because they thought they were not as good at coding as other developers.</p> <hr> <h2 id="patreon-posts-for-may--2023">Patreon Posts For May 2023</h2> <ul> <li><a href="https://www.patreon.com/posts/testerhq-using-83847669">TesterHQ Using Userspice</a></li> </ul> <p>TesterHQ.com was an old domain name that I had floating around for a while and didn&rsquo;t use, so I set it up as a Learning Management System</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/reading-old-code-83786908">Reading Old Code</a></li> </ul> <p>Old code can be hard to read, refactor it and remove code that isn&rsquo;t used.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/bend-rules-83544871">Bend the rules</a></li> </ul> <p>Bending the rules for process change.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/userspice-83437288">UserSpice</a></li> </ul> <p>Adding user authentication and management to TesterHQ</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/audio-version-83327644">Audio Version - The Test Entities Show - Test Cases and Test Scripts</a></li> </ul> <p>The most common Testing Entities that I&rsquo;ve encountered over the years the audio podcast version.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/test-entities-83327781">The Test Entities Show - video and show notes</a></li> </ul> <p>The most common Testing Entities that I&rsquo;ve encountered over the years discussed as video with show notes.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/tools-used-for-83290379">Tools Used for Next Podcast</a></li> </ul> <p>I&rsquo;ve just recorded a new podcast episode based on an April Patreon post usingObsidian Canvas and other tools.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/jupyter-in-83196976">Jupyter in docker</a></li> </ul> <p>I wanted to experiment with Jupyter labs and had to use docker to do so.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/evil-tester-show-82946825">The Evil Tester Show - Manual Testing Episode - Audio</a></li> </ul> <p>The Evil Tester Show, Manual Testing Episode - audio only version.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/manual-testing-82946762">The Manual Testing Episode</a></li> </ul> <p>The next episode of The Evil Tester Show - ad free video version.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/creating-custom-82921942">Creating Custom Images and Containers</a></li> </ul> <p>Lessons learned from creating a custom image.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/find-watch-top-82753559">find, watch, top</a></li> </ul> <p>a few useful linux commands in context</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/using-docker-82644761">Using Docker images - example Gruyere</a></li> </ul> <p>Example of using a Gruyere docker image.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/docker-mini-tips-82501391">Docker mini tips</a></li> </ul> <p>Some tips I wrote for myself when learning docker.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/initial-docker-i-82399583">The Initial Docker Notes I needed</a></li> </ul> <p>My notes on docker after working through my self-study questions list.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/going-back-to-82330520">Going back to basics</a></li> </ul> <p>Going back to learn the basics of Unix CLI usage.</p> <hr> <p><em>You can gain access to the content on Patreon for only $1 a month at <a href="https://patreon.com/eviltester">patreon.com/eviltester</a>. Supporters gain access to downloadable PDFs of the content and micro courses e.g. <a href="https://www.eviltester.com/page/onlinetraining/howitestweb-patreon/">How I Test - Web App</a>, <a href="https://www.eviltester.com/page/onlinetraining/testabilityaudit/">Test Ability Audit</a> and <a href="https://www.eviltester.com/page/patreon/#microcourses">more</a></em></p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Episode 019 - The Test Cases and Test Scripts Episode - The Evil Tester Show https://www.eviltester.com/show/019-test-entitites-cases-scripts/ Sun, 04 Jun 2023 07:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/show/019-test-entitites-cases-scripts/ <p>The most common Testing Entities that I&rsquo;ve encountered over the years are: Test Condition, Test Scenario, Test Case, Test Script. In my Testing I pretty much now have: Test Idea, Test Log (Execution Log). Is there any value in the original Testing Entities? There might be if we view them as logical concepts, but not really as physical artifacts.</p> <p>This was released to <a href="https://patreon.com/eviltester">Patreon supporters</a> early, with full transcript and no ads.</p> <h2 id="test-entities">Test Entities</h2> <p>The closest to &ldquo;Test Script&rdquo; is that I would map that on to an <code>@Test</code> annotated method for automating the execution and assertion of specific conditions.</p> <p><img src="https://www.eviltester.com/images/podcast/019-images/test-entities.png" alt=""></p> <h2 id="legacy-start-point">Legacy Start Point</h2> <p>When I started in testing I took it for granted that professional testing involved using all the entities, and this was usually enforced by the tooling.</p> <p>There would be a Requirements Tracking style hierarchy that the Test Conditions and Test Scenarios would be part of, and these would have cross references back to requirement documentation.</p> <p>Some tools had plugins so that if you were reading a requirement document in MS Word, highlight text and then mark it as a Test Condition or Test Requirement and it would be populated in your tool.</p> <p>This tight level of coupling between tools and concepts made changes incredibly hard to manage and very time consuming.</p> <p>Test Cases might be organised into hierarchies and they would be cross referenced to Test Conditions to justify their existence.</p> <p>Test Cases would then be organised into Test Suites to plan the testing.</p> <p>Each Test Case would have to have a Test Script in order to be &rsquo;executed&rsquo; and tracking of testing was all done by marking Test Scripts as executed or not.</p> <p>I don&rsquo;t know which tools still enforce this type of behaviour and I see online people talking about Test Cases and showing what I thought of as Test Scripts.</p> <h2 id="logical-expansion">Logical Expansion</h2> <p><img src="https://www.eviltester.com/images/podcast/019-images/test-entities-as-logical-concepts.png" alt=""></p> <p>In one sense the entities:</p> <ul> <li>Test Condition</li> <li>Test Scenario</li> <li>Test Case</li> <li>Test Script</li> </ul> <p>These are:</p> <ul> <li>Derivational (Derived From)</li> <li>Expansion of Previous</li> </ul> <p>As Logical Concepts they are fine. As Physical Implementations they will impact effective testing</p> <p>They can be viewed as basically the same thing, but at each level they become more concrete or physical.</p> <h3 id="test-condition">Test Condition</h3> <ul> <li>Something I&rsquo;m going to use as a basis for my testing</li> <li>Input</li> <li>Process</li> <li>Output</li> <li>Derived from External Model</li> </ul> <h3 id="test-scenario">Test Scenario</h3> <ul> <li>A collection of Test Conditions</li> <li>A Test Condition</li> </ul> <h3 id="test-case">Test Case</h3> <ul> <li>A specific set of inputs</li> <li>A specific process</li> <li>A set of expected result</li> <li>Some people combine this with Test Script</li> </ul> <p>Each of the above from either Test Conditions or Test Scenarios</p> <p>A Test Case is a collation of Test Conditions as they map on to the system rather than the requirements.</p> <h2 id="test-script">Test Script</h2> <ul> <li>A set of steps to follow</li> <li>A set of interrogations to make</li> </ul> <p>A Test Script is the physical representation of the Logical &rsquo;thing we want to achieve&rsquo; (Test Case). It is the &lsquo;how we want to achieve it&rsquo; (Test Script).</p> <p>I learned that Test Scripts were too hard to maintain and really if I knew how to use the system I didn&rsquo;t need a detailed Test Script, I could work from a Test Case because I knew how to operate the system I didn&rsquo;t need a list of detailed instructions for each Test Case.</p> <p>Sometimes a list of instructions can be useful, but it would be a template, it wouldn&rsquo;t dictate specific data.</p> <p>And the more I knew the system the less detailed the Test Cases needed to be because I just needed a prompt about what I needed to test, what specific idea or condition I was trying to test.</p> <h2 id="work-from-ideas">Work from ideas</h2> <p>Eventually that turned into a list of &lsquo;ideas&rsquo; and it was more important to document:</p> <ul> <li>what I think I&rsquo;m going to do and why</li> <li>what I actually did and how</li> <li>what I think I still have to</li> </ul> <h2 id="as-physical-concepts">As Physical Concepts:</h2> <p><img src="https://www.eviltester.com/images/podcast/019-images/test-entities-as-physical-concepts.png" alt=""></p> <p>This is where Testing falls apart, becomes:</p> <ul> <li>highly inefficient,</li> <li>time consuming,</li> <li>hard to maintain,</li> <li>coverage is based on scripts or cases</li> </ul> <p>Formalised as:</p> <ul> <li>spreadsheets,</li> <li>templates,</li> <li>tools</li> </ul> <p>Because of where you work you may &lsquo;have&rsquo; to use these.</p> <p>Testing does not require the use of these as physical implementations.</p> <h2 id="keep-them-logical">Keep Them Logical</h2> <p>Very detailed representations of these entities makes the process of Software Testing incredibly time consuming.</p> <p>Imagine&hellip; you get a new list of requirements which are documented. You now have to update all the existing conditions, create new conditions, cross reference them to existing or new Test Cases. If the docs list changes to the physical implementation then that means updating the Test Scripts&hellip; and there was no &lsquo;find and replace&rsquo; mechanism.</p> <p>Then when you actually started following the Test Scripts, you would get to step 2, discover the system didn&rsquo;t match the script, then have to go off and update all the test scripts so you didn&rsquo;t make much progress.</p> <p>I think the general concepts behind the entiies are fine:</p> <ul> <li>we have to know why we are doing something</li> <li>we want to know what we are trying to achieve and what distinguishes it from other tests we&rsquo;ve done</li> <li>we need to know how to use the software and spot issues</li> </ul> <p>Codifying these concepts into specific entities is good for Tool Vendors, and for expensive outsourced projects, it isn&rsquo;t good for the cost conscious project or for testing.</p> <h2 id="concepts">Concepts</h2> <p>If we know the concepts behind the entities then we can more creatively work in environments that force us to use tools that implement the concepts.</p> <p>Test Scripts become execution logs where the &lsquo;actual results&rsquo; are where we write our logs. The &rsquo;expected results&rsquo; are often &lsquo;-&rsquo; because they probably made the field mandatory.</p> <p>But it is still very painful to split your list of ideas into separate Test Cases.</p> <p>I tend to review the testing I&rsquo;ve done as &ldquo;Tests&rdquo; rather than &ldquo;Test Cases&rdquo;.</p> <p><img src="https://www.eviltester.com/images/podcast/019-images/how-do-test-entities-help.png" alt=""></p> <h3 id="test-condition---ideas">Test Condition - Ideas</h3> <ul> <li>Something I&rsquo;m going to use as a basis for my testing</li> <li>Thinking in advance about what I want to test for</li> <li>Input, Process, Output</li> <li>Derived from External Model</li> <li>Test Ideas</li> </ul> <h3 id="test-scenario---process-ideas">Test Scenario - Process ideas</h3> <ul> <li>A process oriented test idea</li> <li>What if&hellip; X and Y happen</li> </ul> <h3 id="test-case---specifics">Test Case - Specifics</h3> <ul> <li>What specifically is important?</li> <li>Why specifically?</li> <li>What are my oracles?</li> <li>How deep do I want to Interrogate?</li> </ul> <h3 id="test-script-1">Test Script</h3> <p>Test Script doesn&rsquo;t actually tell me the things below, but that&rsquo;s what this concept reminds me that I need:</p> <ul> <li>Do I know how to use the system?</li> <li>Do I know why the system is used?</li> <li>Do I know what the functionality is for?</li> <li>Do I know how to interrogate the system?</li> <li>Do I know how to admin the system?</li> </ul> <p>Also:</p> <ul> <li>Set of instructions for complicated actions</li> <li>Can I automate?</li> <li>Execution templates</li> <li>What physical path based model am I using?</li> <li>Physical Path based coverage</li> </ul> <p>I don&rsquo;t see a lot of value in the use of the entities any more. But I still find some value in the concepts that underpin them.</p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Episode 018 - The Manual Testing Special - The Evil Tester Show https://www.eviltester.com/show/018-manual-testing/ Sun, 21 May 2023 07:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/show/018-manual-testing/ <p>Top 3 Phrases that should carry trigger warnings for the Test Community: Manual Testing, Test Automation, Quality Assurance. In this episode I&rsquo;m going to talk about Manual Testing. And I&rsquo;m going to talk about what we might want to say instead of &ldquo;Manual Testing&rdquo;.</p> <p>This was released to <a href="https://patreon.com/eviltester">Patreon supporters</a> early, with full transcript and no ads.</p> <h2 id="one-thing-in-common">One Thing in Common</h2> <p>And they all have something else in common.</p> <p>They are all noun phrases.</p> <p>I try to think in &ldquo;Verb noun&rdquo; phrases to keep communication simpler and less vague.</p> <p>I don&rsquo;t want to police the language that other people use, I want to police the language I use to try to be as clear as I can when I communicate.</p> <p>A heuristic for spotting Noun Phrases.</p> <ul> <li>if a phrase X describes something that you do, but in order to use it you have to say &ldquo;I&rsquo;m doing X&rdquo; then it&rsquo;s probably a noun phrase.</li> </ul> <p>I&rsquo;m &ldquo;Doing the dishes&rdquo; instead of &ldquo;I&rsquo;m washing the dishes&rdquo;.</p> <p>Washing the dishes is a verb phrase that describes the process.</p> <h2 id="but-its-all-testing-just-say-testing">But it&rsquo;s all Testing, Just say Testing</h2> <p>It is all Testing, but if I just say &ldquo;I&rsquo;m Testing&rdquo; then I&rsquo;m hiding a lot of the nuances about what I&rsquo;m doing.</p> <ul> <li>Testing (verb)</li> <li>Manually</li> <li>By Hand</li> <li>For Performance characteristics</li> <li>While Automating</li> <li>From Hand drawn state charts</li> <li>Old bugs that we think we have fixed</li> <li>etc.</li> </ul> <p>With a verb phrase I can add all the additional descriptive words after testing. And manually seems to fit well OK there.</p> <p>But I do have a lot of sympathy with this view of &ldquo;Just say Testing&rdquo; because most of the time when I see videos or blog post about &ldquo;Manual Testing&rdquo; they are really just describing a generic testing process.</p> <p>They are not describing Testing Manually at all.</p> <p>And they are not even describing how they test, it&rsquo;s just a generic: Start with requirements, write some scenarios, blah de blah.</p> <p>It&rsquo;s possibly good that they don&rsquo;t describe it as &ldquo;Testing&rdquo; because what they are describing is a horrible approach to collaborative Software Testing on a project and is more like a model of &ldquo;Expensive Outsourced Testing&rdquo;.</p> <p>So maybe say that instead &ldquo;Expensive Outsourced Testing&rdquo;</p> <ul> <li>&ldquo;what are you doing?&rdquo;</li> <li>&ldquo;Oh, I&rsquo;m just doing some Expensive Outsource Testing&rdquo;</li> <li>No no don&rsquo;t do that, can&rsquo;t you just test it manually instead?</li> </ul> <h2 id="but-we-dont-say-manual-programming">But we don&rsquo;t say Manual Programming</h2> <p>No, because programming is a verb. And we haven&rsquo;t really automated it very well.</p> <p>But people do now say&hellip; :</p> <ul> <li>I&rsquo;m programming</li> <li>using ChatGPT</li> <li>using Co-pilot</li> </ul> <h2 id="language-is-ambiguous">Language is Ambiguous</h2> <p>Language is ambiguous.</p> <p>Particularly when we have words that can be used as both nouns and verbs, depending on the context.</p> <p>Words like &ldquo;Testing&rdquo; and &ldquo;Programming&rdquo;.</p> <p>To demonstrate that usage context matters, I fed different prompts into Chat GPT.</p> <p>Asking a very limited question which has few options for the answer, I received the answer I was looking for.</p> <pre tabindex="0"><code>&gt; Is this a noun phrase or a verb phrase: &#34;manual testing&#34; &#34;Manual testing&#34; is a noun phrase. It consists of the noun &#34;testing&#34; modified by the adjective &#34;manual.&#34; &gt; Is this a noun phrase or a verb phrase: &#34;testing manually&#34;? &#34;Testing manually&#34; is a verb phrase. It consists of the verb &#34;testing&#34; modified by the adverb &#34;manually.&#34; </code></pre><p>But if I ask a more general question then I receive options that explore the variety of usages.</p> <hr> <pre tabindex="0"><code>&gt; Is Manual Testing a Noun Phrase? </code></pre><p>Yes, &ldquo;Manual Testing&rdquo; can be considered a noun phrase. A noun phrase is a group of words centered around a noun that functions as a single unit and can act as a subject, object, or complement in a sentence. In the case of &ldquo;Manual Testing,&rdquo; the noun is &ldquo;Testing,&rdquo; and &ldquo;Manual&rdquo; is an adjective modifying the noun. Together, they form a noun phrase that represents a specific type or approach to testing that is done manually, without the use of automated tools.</p> <hr> <pre tabindex="0"><code>&gt; Is Manual Testing a Noun Phrase? </code></pre><p>Yes, &ldquo;Manual Testing&rdquo; is a noun phrase. A noun phrase is a group of words centered around a noun that functions as a noun in a sentence. In this case, &ldquo;Manual Testing&rdquo; consists of the noun &ldquo;Testing&rdquo; modified by the adjective &ldquo;Manual,&rdquo; indicating that the testing is done by hand or manually.</p> <hr> <pre tabindex="0"><code>&gt; Is Manual Testing a Noun Phrase? </code></pre><p>No, &ldquo;Manual Testing&rdquo; is not a noun phrase. It is a noun followed by a present participle acting as an adjective. In this case, &ldquo;manual&rdquo; describes the type or method of testing being performed, while &ldquo;testing&rdquo; is the noun representing the action or process of testing.</p> <hr> <pre tabindex="0"><code>&gt; Is Manual Testing a Noun Phrase? </code></pre><p>No, &ldquo;Manual Testing&rdquo; is not a noun phrase. It is a compound noun formed by combining the adjective &ldquo;manual&rdquo; and the noun &ldquo;testing.&rdquo; A noun phrase is a group of words centered around a noun that functions as a subject, object, or complement in a sentence. For example, in the sentence &ldquo;I performed manual testing,&rdquo; &ldquo;manual testing&rdquo; is a noun phrase where &ldquo;testing&rdquo; is the noun and &ldquo;manual&rdquo; is an adjective describing the type of testing being performed.</p> <hr> <pre tabindex="0"><code>&gt; Is Manual Testing a Noun Phrase? </code></pre><p><em>This was my particular favourite answer where it is not a noun phrase because it is a noun phrase.</em></p> <p>No, &ldquo;Manual Testing&rdquo; is not a noun phrase. It is a noun followed by a present participle, which forms a noun phrase. &ldquo;Manual&rdquo; is an adjective modifying the noun &ldquo;Testing,&rdquo; which is the gerund form of the verb &ldquo;test.&rdquo; Together, they form the noun phrase &ldquo;Manual Testing.&rdquo;</p> <p>By describing the process we use, at the current point in time, with the current aims we are attempting to achieve, we can reduce the ambiguity in our communication.</p> <h2 id="adverbs-help-me-focus">Adverbs Help me focus</h2> <p>Adverbs help me focus and avoid my biases.</p> <ul> <li>I&rsquo;d avoid doing &ldquo;Manual Testing&rdquo;, instead &ldquo;I&rsquo;m testing this manually, I haven&rsquo;t decided if I will automate this yet&rdquo;</li> </ul> <p>I do have the word &ldquo;manually&rdquo; in my vocabulary when I think I need to explicitly state that I am not automating something.</p> <p>I use the word for clarity, but it is an adverb rather than making a nominalisation out of &ldquo;Testing&rdquo;.</p> <p>I mostly use it silently when Thinking.</p> <p>I use it when thinking about what I&rsquo;m going to do because I might jump to &lsquo;automating&rsquo;, particularly when I&rsquo;m working in a code base with a lot of Automated Execution coverage already in place.</p> <p>I sometimes jump into very technical Tool based interrogation testing too quickly. So &ldquo;manual&rdquo; helps me focus on higher level coverage because I can&rsquo;t go as deep into the system as quickly without tools.</p> <p>I might use the word manually in response to a call for clarity e.g.</p> <pre tabindex="0"><code>Alan: I&#39;m testing the file processing we just introduced. Person: Are you automating it? Alan: No, I&#39;m testing it manually Alan-Earth-2: No, I&#39;m testing it by hand </code></pre><p>At other times I might say &ldquo;No, I&rsquo;m testing it by hand&rdquo;, I&rsquo;m adding the adverb for clarity to distinguish it from the process of automating.</p> <p>My language choice means that I don&rsquo;t tend to say &ldquo;Manual Testing&rdquo; but I also don&rsquo;t feel a strong aversion to it.</p> <p>I use this to help me. And it does help me.</p> <p>I have a bigger aversion to the use of Quality Assurance than I have to Manual Testing but different words trigger different people and I find that talking in terms or processes and in more detail about how we do what we do helps improve communication.</p> <p>And always remember, it could be worse. People might talk about &ldquo;Manual QA&rdquo; instead.</p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Applications to Practice Testing and Automating https://www.eviltester.com/post/applications-to-practice-testing-and-automating/ Thu, 02 Mar 2023 12:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/post/applications-to-practice-testing-and-automating/ <p>TLDR; There is no need to practice testing on public websites and annoy the owners. There are lots of applications to use to practice.</p> <p>Back in <a href="https://twitter.com/eviltester/status/1497277397325996032">February of 2022</a> I posted a Twitter thread with a list of practice applications. Now I have converted the thread to a blog post with additional information.</p> <h2 id="the-compendium-of-testing-apps">The Compendium of Testing Apps</h2> <p>Many of the applications below have been collated into a Compendium of testing Apps.</p> <ul> <li><a href="https://www.eviltester.com/page/tools/compendiumtesting/">The Compendium of Testing Apps</a></li> </ul> <p>This contains, The Pulper, Test Pages and the buggy games.</p> <p>The above page contains overview videos explaining the application and how to use it to practice.</p> <h2 id="the-pulper">The Pulper</h2> <p><a href="https://thepulper.herokuapp.com/">The Pulper</a> is a full CRUD application with an API. Check out the Admin menu and you&rsquo;ll see there are multiple versions to test. Oh, and there is a REST API too.</p> <p>Learn more: <a href="https://www.eviltester.com/page/tools/thepulper">eviltester.com/page/tools/thepulper</a></p> <p>I also created a <a href="https://www.testerhq.com/member2/index.php?course=the_pulper_technical_testing_workshop&amp;page=coursecontents&amp;preview=true">short online course</a> with a <a href="https://www.patreon.com/posts/pulper-self-43748755">113 page Workbook</a> to help with practice sessions. These are available to Patreon supporters as part of their <a href="https://www.patreon.com/eviltester">Patreon membership</a>.</p> <h2 id="rest-api-challenges">Rest API Challenges</h2> <p><a href="https://apichallenges.herokuapp.com/">Challenges</a> for you to complete on a live application with a REST API. There is a GUI to view the data but you won&rsquo;t need that becuase&hellip; you&rsquo;ll use the REST API. A full set of tutorial videos to learn from.</p> <p>Learn More: <a href="https://www.eviltester.com/page/tools/apichallenges/">eviltester.com/page/tools/apichallenges</a></p> <p>This is also supported by an <a href="https://www.testerhq.com/member2/index.php?course=api_challenges&amp;page=coursecontents&amp;preview=true">online course</a> available to Patreon supporters.</p> <h2 id="the-test-pages">The Test Pages</h2> <p><a href="https://testpages.herokuapp.com/styled/index.html">The Test Pages</a> are a set of online pages to support both Testing and Automating. They don&rsquo;t come with a lot of instructions, but part of the fun is making your own exercises.</p> <p>Learn More: <a href="https://www.eviltester.com/page/tools/testpages/">eviltester.com/page/tools/testpages</a></p> <h2 id="the-todo-list">The Todo List</h2> <p><a href="https://eviltester.github.io/simpletodolist/todolists.html">The Todo List</a> is based on the <a href="https://todomvc.com/examples/vanillajs/">Vanilla JS Todo MVC</a> application, but with more functionality.</p> <p>Learn More: <a href="https://www.eviltester.com/page/tools/thetodoapp/">eviltester.com/page/tools/thetodoapp/</a></p> <p>This application was used in my LinkedIn Learning Course <a href="https://www.eviltester.com/page/onlinetraining/courses/#selenium-webdriver-page-objects-and-abstractions">WebDriver Page Objects and Abstractions</a></p> <h2 id="7-char-validation">7 Char Validation</h2> <p><a href="https://eviltester.github.io/TestingApp/apps/7charval/simple7charvalidation.htm">7 Char Validation</a> is the first &lsquo;designed for testing&rsquo; application I created.</p> <p>It seems very simple, with two input fields, but I recently revisited it to create additional exercises and now there is a <a href="https://www.patreon.com/posts/75068248">200+ Page workbook</a> filled with testing exercises using only the 7 Char Validation app. This workbook is available to Patreon Subscribers as part of their membership.</p> <h2 id="protect-the-square">Protect The Square</h2> <p><a href="https://eviltester.github.io/TestingApp/games/buggygames/protect_the_square/protect_the_square.html">Protect The Square</a> is part of the <a href="https://www.eviltester.com/page/tools/buggygames/">Buggy Games</a> Testing Apps.</p> <p>Learn more: <a href="https://www.eviltester.com/page/onlinetraining/protectthesquare/">about the course</a> I created a short <a href="https://www.testerhq.com/member2/index.php?course=protect_the_square&amp;page=coursecontents&amp;preview=true">online course</a> to explain various techniques and exercises using the application. The course and accompanying workbook are available to Patreon supporters.</p> <h2 id="selected-examples">Selected Examples</h2> <p>From <a href="https://www.eviltester.com/page/tools/testpages/">The Test Pages</a>:</p> <ul> <li><a href="https://testpages.herokuapp.com/styled/validation/input-validation.html">Input Validation Examples</a></li> <li><a href="https://testpages.herokuapp.com/styled/calculator">Simple Calculator</a></li> <li><a href="https://testpages.herokuapp.com/styled/apps/calculator.html">Button Based Calculator</a></li> <li><a href="https://testpages.herokuapp.com/styled/apps/triangle/triangle001.html">The Triangle Problem</a></li> <li><a href="https://testpages.herokuapp.com/styled/gui-scribble.html">Canvas Scribble</a></li> <li><a href="https://testpages.herokuapp.com/styled/canvas-javascript-test.html">Cavas Drawing</a></li> <li><a href="https://testpages.herokuapp.com/styled/javascript-countdown-test.html">Javascript counter</a></li> <li><a href="https://eviltester.github.io/TestingApp/apps/index.html">The Buggy Apps</a> 8 apps with multiple versions</li> <li><a href="https://eviltester.github.io/TestingApp/apps/counterstrings/counterstrings.html">CounterStrings</a></li> </ul> <h2 id="buggy-games">Buggy Games</h2> <p>The <a href="https://eviltester.github.io/TestingApp/games/index.html">Buggy Games section</a> has 10 Buggy games with multiple versions for you to test your skills against.</p> <h2 id="blog-posts-and-videos">Blog Posts and Videos</h2> <p>You might also find these blog posts and videos useful:</p> <ul> <li><a href="https://www.eviltester.com/2016/12/a-testing-challenge-for-you-i-test-with.html">I Test with Attitude</a></li> <li><a href="https://www.eviltester.com/2018/05/protect-the-square.html">Protect the Square and Buggy Games</a></li> </ul>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; February 2023 EvilTester.com and Patreon Content Summary https://www.eviltester.com/blog/eviltester/summaries/2023-02-february-2023-summary/ Wed, 01 Mar 2023 06:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/summaries/2023-02-february-2023-summary/ <p>Content Summary for February 2023 EvilTester.com and Patreon.</p> <p><a href="https://www.patreon.com/posts/24389995">All monthly summaries of Patreon and blog content</a></p> <h2 id="patreon-posts-for-february--2023">Patreon Posts For February 2023</h2> <ul> <li><a href="https://www.patreon.com/posts/integration-79406036">Integration Testing</a></li> </ul> <p>Trying to resolve differences between Automated Unit test behaviour in different environments.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/animated-cli-79290562">Animated Graphviz CLI</a></li> </ul> <p>I&rsquo;ve just created a new repo on Github. This has the generate animated Graphviz cli</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/docker-docs-79157420">Docker Docs</a></li> </ul> <p>Most official documentation for tooling is quite poor.I can however, thoroughly recommend the documentation for Docker.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/parameterized-78960028">Parameterized Automated Execution</a></li> </ul> <p>One low cost way I like to experiment with unit tests is by adding some adhoc parameterisation.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/automation-roi-78833345">Automation ROI</a></li> </ul> <p>I saw a post on linkedin on calculating Test Automation ROI&hellip;</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/randomness-78785255">Randomness</a></li> </ul> <p>I like to use random data in my automated execution&hellip; why?</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/annotations-78736299">Annotations</a></li> </ul> <p>I&rsquo;m working on a Spring Boot application at the moment &hellip; so many annoations.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/secure-coding-78638854">Secure Coding Fundamentals Talk</a></li> </ul> <p>A good video on Secure Coding Fundamentals.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/confidence-and-78506970">Confidence and competence comes from practice</a></li> </ul> <p>Confidence and competence comes from practice. There is no magic. Just practice.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/management-78369737">Management Theory</a></li> </ul> <p>I read through &lsquo;Management Theory - From Taylorism to Japanization&rsquo; by John Sheldrake</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/feb-9th-twitter-78315684">Feb 9th Twitter API changes might mean death of Chatterscan</a></li> </ul> <p>On Feb 9th 2023 Chatterscan will probably die when twitter stop making their api free. Considering some options.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/triggering-to-78179400">Triggering ffmpeg to create mp4</a></li> </ul> <p>I thought that it would be easier to use a library to control ffpmeg.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/auto-animating-78079815">Auto Animating Graphviz</a></li> </ul> <p>I&rsquo;ve been experimenting with different ways to try and create an animated version of the Graphviz rendering programmatically.</p> <hr> <p><em>You can gain access to the content on Patreon for only $1 a month at <a href="https://patreon.com/eviltester">patreon.com/eviltester</a>. Supporters gain access to downloadable PDFs of the content and micro courses e.g. <a href="https://www.eviltester.com/page/onlinetraining/howitestweb-patreon/">How I Test - Web App</a>, <a href="https://www.eviltester.com/page/onlinetraining/testabilityaudit/">Test Ability Audit</a> and <a href="https://www.eviltester.com/page/patreon/#microcourses">more</a></em></p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; January 2023 EvilTester.com and Patreon Content Summary https://www.eviltester.com/blog/eviltester/summaries/2023-01-january-2023-summary/ Wed, 01 Feb 2023 12:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/summaries/2023-01-january-2023-summary/ <p>Content Summary for January 2023 EvilTester.com and Patreon.</p> <p><a href="https://www.patreon.com/posts/24389995">All monthly summaries of Patreon and blog content</a></p> <h2 id="blog-posts-for-january--2023">Blog Posts For January 2023</h2> <ul> <li><a href="https://eviltester.com/post/fundamentals/a-meta-model">What is Software Testing? - a meta model</a></li> </ul> <p>A meta model of software testing built from first principles.</p> <h2 id="patreon-posts-for-january--2023">Patreon Posts For January 2023</h2> <ul> <li><a href="https://www.patreon.com/posts/from-taylorism-77962118">From Taylorism to Fordism</a></li> </ul> <p>Thoughts on automating after reading &lsquo;From Taylorism to Fordism&rsquo;</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/source-is-how-77827253">Source is How, Docs are What</a></li> </ul> <p>I dislike writing documentation for my own programs but docs explain the &lsquo;what&rsquo;, &lsquo;why&rsquo;, &lsquo;where&rsquo; and &lsquo;when&rsquo;. The source can detail the &lsquo;how&rsquo;.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/published-to-npm-77639583">published to npm and cdn</a></li> </ul> <p>24 hours after messing up my npm publish I was finally able to publish to npm and cdn</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-2022-77589161">Summary for December 2022</a></li> </ul> <p>This is a list of content I posted in December 2022.Back to Index of all monthly summariesPatreon Posts For December 2022Visual Studio Code</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/summary-for-2022-77589065">Summary for November 2022</a></li> </ul> <p>This is a list of content I posted in November 2022.Back to Index of all monthly summariesPatreon Posts For November 2022Patching and re-wri</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/december-2022-77588977">December 2022 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon and blog content for December 2022 is attached to this post.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/november-2022-77588895">November 2022 Collated PDFs</a></li> </ul> <p>The PDF for all Patreon and blog content for November 2022 is attached to this post.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/creating-npm-77584671">Creating an npm package</a></li> </ul> <p>My first attempt at releasing a package to npm</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/research-during-77465683">Research during learning</a></li> </ul> <p>When I&rsquo;m learning a topic I tend to overload myself with information&hellip;</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/how-to-move-with-77371462">How to move forward with Javascript Automating?</a></li> </ul> <p>I was asked a question today about how to move forward from manually exploring APIs in Postman to automating using JavaScript.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/animated-editor-77230602">Animated Graphviz Editor and Docs</a></li> </ul> <p>Created a very basic editor for Animated Graphviz with some docs.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/webpack-to-js-77060522">Webpack to create JS library for browser</a></li> </ul> <p>I got the animated Graphviz parsing working in the browser.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/typescript-poc-77005188">TypeScript POC to create animated Graphviz</a></li> </ul> <p>I wanted a small project to practice TypeScript with.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/test-behaviour-76884303">Test Behaviour not Implementation</a></li> </ul> <p>Impact of testing implementation not behaviour.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/narrated-meta-76845039">Narrated Meta-model</a></li> </ul> <p>One thing I&rsquo;m trying to tdo this year is create more videos and public content.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/how-to-create-76729382">How To - create animated Graphviz Diagram</a></li> </ul> <p>To create the video for the animated Software Testing Meta Model I used Graphviz.Specifically, d3-graphviz. Here is how&hellip;</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/animated-testing-76676071">Animated Software Testing Metamodel with Explanatory Text</a></li> </ul> <p>An attempt at exploring a meta-model or high-level generalised model of Software Testing to explain it in &rsquo;normal&rsquo; language.</p> <hr> <ul> <li><a href="https://www.patreon.com/posts/more-graphviz-76636697">More Graphviz Visual Studio Code Plugins</a></li> </ul> <p>Unfortunately the Graphviz plugin that I was using stopped working for my diagram. So I tried another.</p> <hr> <p><em>You can gain access to the content on Patreon for only $1 a month at <a href="https://patreon.com/eviltester">patreon.com/eviltester</a>. Supporters gain access to downloadable PDFs of the content and micro courses e.g. <a href="https://www.eviltester.com/page/onlinetraining/howitestweb-patreon/">How I Test - Web App</a>, <a href="https://www.eviltester.com/page/onlinetraining/testabilityaudit/">Test Ability Audit</a> and <a href="https://www.eviltester.com/page/patreon/#microcourses">more</a></em></p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; What is Software Testing? - a meta model https://www.eviltester.com/post/fundamentals/a-meta-model/ Sun, 22 Jan 2023 09:30:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/post/fundamentals/a-meta-model/ <p>TLDR; Modelling is a fundamental skill for Software Testers. Applying modelling to Software Testing itself allows us to communicate our testing in simple terms and we can explore how concepts build on, and relate to, each other.</p> <h2 id="video">Video</h2> <div class="video-container"> <iframe width="560" height="315" src="https://www.youtube.com/embed/x9c3ZTG7Ub4" allow="autoplay; encrypted-media" allowfullscreen></iframe> </div> <div><p class="center-text"><a href="https://www.youtube.com/watch?v=x9c3ZTG7Ub4" target="_blank">Watch on YouTube</a></p></div> <h2 id="fundamentals">Fundamentals</h2> <p>Testing is underpinned by models.</p> <p>e.g. requirements, user stories, scenarios, etc.</p> <div> <p> <img class="image-center" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0000.dot.png" alt=""> </p> </div> <p>We have the &rsquo;thing we are comparing with&rsquo;. Usually a System Under Test.</p> <div> <p> <img class="image-center" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0001.dot.png" alt=""> </p> </div> <p>The model and the SUT are inputs to our testing process.</p> <div> <p> <img class="image-center" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0002.dot.png" alt=""> </p> </div> <h2 id="evaluating-by-comparing">Evaluating by Comparing</h2> <p>The simplest way to use a model for testing is to compare the model with the System Under Test.</p> <div> <p> <img class="image-center" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0004.dot.png" alt=""> </p> </div> <p>This generates output results from the comparison.</p> <div> <p> <img class="image-center" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0005.dot.png" alt=""> </p> </div> <p>Evidential Observation Output can be used for negative or positive confirmation.</p> <div> <p> <img class="image-center" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0009.dot.png" alt=""> </p> </div> <p>Positive confirmation is often viewed as a &lsquo;Test Pass&rsquo;. And basically means that &lsquo;when we did something we observed that the system behaved the way that we expect&rsquo;.</p> <p>NOTE: this does not mean that the system &ldquo;works&rdquo;, it just means that at some point in time, given a specific set of input and process we observed that the system behaaved as we expected.</p> <p>Negative Confirmation is often called a &lsquo;Test Fail&rsquo;.</p> <p>This means we observed something we did not expect.</p> <p>These positive and negative reports are a visible output from the Testing process.</p> <h2 id="investigate-information">Investigate Information</h2> <p>So we have to investigate what failed during our comparison process.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0010.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0010.dot.png" alt=""></a> </p> </div> <p>This might mean:</p> <ul> <li>a problem with our model</li> <li>a problem with the System Under Test</li> <li>a problem with our comparison process</li> </ul> <p>Either way it is something we have to investigate.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0013.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0013.dot.png" alt=""></a> </p> </div> <h2 id="information-leads-to-change">Information Leads to Change</h2> <p>System Under Test changes might require a defect process. Rather than a direct system fix.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0014.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0014.dot.png" alt=""></a> </p> </div> <p>A comparison process from model to &rsquo;thing&rsquo; isn&rsquo;t the only process that Software Testing involves, it&rsquo;s just one of the most obvious ways we have of evaluating the Software.</p> <p>And Software Testing is a process of evaluating the software.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0015.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0015.dot.png" alt=""></a> </p> </div> <p>We can directly compare Requirements with the Software for high level surface positive or negative confirmation that a requirement has been implemented.</p> <p>e.g. &ldquo;A User must be able to login with their username and password.&rdquo; can be confirmed quite simply by logging in with an existing user and their correct password.</p> <h2 id="model-exploration">Model Exploration</h2> <p>But during the Software Testing process we also try to expand our model beyond Requirements.</p> <p>We also explore our model in conjunction with the system to learn more about our model and as a consequence, the system itself.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0016.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0016.dot.png" alt=""></a> </p> </div> <p>Software Testing has to explore our model to identify:</p> <ul> <li>risks</li> <li>ambiguity</li> <li>gaps in the models and implementation</li> <li>etc.</li> </ul> <p>This raises questions which we have to investigate and discuss.</p> <p>The ongoing process of modelling is how we generate our ideas for testing and how we find new ways to explore the software.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0017.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0017.dot.png" alt=""></a> </p> </div> <p>The model is both an input to testing, and an output from Testing.</p> <p>Parts of the model will be stored in the Tester&rsquo;s head as a mental model.</p> <p>Other parts will be visible in the form of diagrams, reports, lists of test ideas, etc.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0018.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0018.dot.png" alt=""></a> </p> </div> <p>Software Testing then has to explore the model in conjunction with the system to identify if any of these manifest as issues or potentially require reworking the implementation.</p> <h2 id="a-little-about-ambiguity">A Little About Ambiguity</h2> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0019.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0019.dot.png" alt=""></a> </p> </div> <p>One example of ambiguity arises from requirements being naturally incomplete because of assumptions in understanding.</p> <p>The requirement for logging into the system does not describe:</p> <ul> <li>what happens if the user is already logged in?</li> <li>what message is displayed if the user gets the password wrong?</li> <li>what message should be displayed if the user does not exist?</li> </ul> <p>Part of the Evaluation process that we call Software Testing is about expanding our model so that we can explore the System more thoroughly and provide more information.</p> <p>We also have to go beyond our model of the requirements and identify Risks associated with potential implementation, or the technology used, or the algorithms we have implemented.</p> <p>For example there might be a risk that the password has been stored in plaintext in the database, and an additional risk that the implementation has coded with a &rsquo;like&rsquo; comparison on the password rather than an &rsquo;equals&rsquo; so it might be possible to use wildcards in the password and login knowing only a partial password.</p> <p>This would also lead to risks about the security of the application and SQL Injection.</p> <p>These might not be covered in the Requirement model, and may not be present in every Tester&rsquo;s or Programmer&rsquo;s model of the System because they may not have experience or knowledge about Security Testing.</p> <p>At times there is endless set of possible ways we could expand our model and how we can explore it in conjunction with the system. And one of the key skills of a Tester is knowing when &rsquo;enough is enough&rsquo; and stopping the identification and exploration of possibilities.</p> <h2 id="exploring">Exploring</h2> <p>The act of exploring a model, or using the model as a basis for exploring the system, is the main way that we test softare.</p> <p>Some of the outputs from this process are captured as part of the model itself.</p> <p>Other outputs vary depending on the process used but include artifacts like:</p> <ul> <li>execution or exploration logs</li> <li>coverage reports</li> <li>questions</li> <li>decisions</li> <li>ideas for new exploration approaches</li> <li>issues</li> <li>issue investigations</li> </ul> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0022.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0022.dot.png" alt=""></a> </p> </div> <p>Exploration is usually pursued in small chunks of time to make it easier to document and respond to the information gained during the exploration to make decisions about what to test or investigate next.</p> <div> <p> <a href="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0024.dot.png" target="_blank"><img class="image-responsive-wrap" src="https://www.eviltester.com/images/blog/basics/meta-model/001-meta-model-of-software-testing.test.dot_0024.dot.png" alt=""></a> </p> </div> <h2 id="summary">Summary</h2> <p>This was an iterative build up of a model about Software Testing. Trying to use common terms rather than specialist language. Trying to avoid any specific framework or development methodology.</p> <p>This is &ldquo;a&rdquo; model. Not &ldquo;the&rdquo; model.</p> <p>This was an example of a modelling process, and the communication of a modelling process.</p> <p>For maximum benefit, create your own models of Software Testing.</p> <blockquote> <p>The diagrams were generated using <a href="https://graphviz.org/">Graphviz</a> from a single <a href="https://github.com/eviltester/software-testing-notes/blob/main/graphviz/001-meta-model-of-software-testing.dot">source file</a> using <a href="https://github.com/eviltester/graphviz-steps">graphviz-steps</a>.</p> </blockquote>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt; Chrome Dev Tools Overview 2022 https://www.eviltester.com/blog/eviltester/technical_testing/2022-09-22-chrome-dev-tools-overview/ Thu, 22 Sep 2022 17:00:00 UTC Alan Richardson Alan Richardson https://www.eviltester.com/blog/eviltester/technical_testing/2022-09-22-chrome-dev-tools-overview/ <p><em>TLDR; Browser dev tools change frequently. I can through the tooling from left to right and explore them to learn the capabilities.</em></p> <p>This post provides a high level summary of the Chrome Dev Tools as they were in September 2022.</p> <ul> <li>Supporting <a href="https://www.youtube.com/watch?v=cwLxQ68SLqA">Youtube video overview</a></li> <li><a href="https://www.patreon.com/posts/chrome-dev-tools-72300708">Ad-free Patreon video overview</a></li> </ul> <h2 id="chrome-dev-tools-in-software-testing">Chrome Dev Tools in Software Testing</h2> <p>Browser Developer Tools does not mean they are only usable by Developers. I&rsquo;ve been using them for Web Testing for years and they keep getting better so it is important to periodically review them for new features.</p> <p>Also, different browsers have different tooling capabilities so its worth exploring the dev tools for all the browser you use.</p> <h2 id="the-pulper-sut">The Pulper SUT</h2> <p>In the video I use The Pulper as the System Under Test to demonstrate the dev tools.</p> <p>The easiest way to find The Pulper is to visit EvilTester.com</p> <ul> <li><a href="https://www.eviltester.com/page/tools/thepulper/">https://www.eviltester.com/page/tools/thepulper/</a></li> </ul> <p>There are links to any hosted versions and instructions to run it locally.</p> <p>There is a Patreon exclusive online course on using The Pulper as a practice application with a free 100 page exercise book.</p> <ul> <li><a href="https://www.testerhq.com/member2/index.php?course=the_pulper_technical_testing_workshop&amp;page=coursecontents">https://www.testerhq.com/member2/index.php?course=the_pulper_technical_testing_workshop&amp;page=coursecontents</a></li> <li><a href="https://www.patreon.com/posts/technical-001-43961970">https://www.patreon.com/posts/technical-001-43961970</a></li> </ul> <h2 id="the-dev-tools">The Dev Tools</h2> <p>Access the Dev Tools by using Right Click and Inspect.</p> <p>Or Choose &ldquo;Developer Tools&rdquo; from the &ldquo;More Tools&rdquo; menu in the browser options hamburger menu.</p> <h2 id="element-selector">Element Selector</h2> <p>The first icon in the dev tools tabs is the Element Selector.</p> <p>I most often use this to navigate to a particular element in the &ldquo;Elements&rdquo; tab.</p> <h2 id="mobile-view-device-toolbar">Mobile View Device Toolbar</h2> <p>I think of this as the Mobile or Responsive Testing View.</p> <p>I use it to simulate mobile devices because when I choose one of the predefined devices Chrome changes the &ldquo;user-agent&rdquo; header in all HTTP requests so the server believes we are now using a mobile device to access the application. e.g. I used to use this to gain access to Instagram features that were only available on mobile when I was using my desktop.</p> <p>I also use this view to see how the page looks at various resolution sizes.</p> <p>Additionally I find it useful to help test CSS breakpoints as these are shown at the top of the screen when the &ldquo;Show Media Queries&rdquo; configuration option is set.</p> <h2 id="elements-tab-dom-view">Elements Tab DOM View</h2> <p>The Elements Tab is the DOM view and shows the Document Object Model (DOM) for the page.</p> <p>The DOM is the internal hierarchical representation of the HTML source with any additional elements that have been added by Javascript.</p> <p>The Elements view allows us to navigate the full DOM and see items which might be hidden from the user. Also I can amend the HTML page here, change form validation controls, etc. This is very useful for helping me test server side functionality without having to use an HTTP tool.</p> <p>In the Elements view I like to use the &ldquo;Event Listeners&rdquo; tab to see what additional functionality has been added to the page through Javascript. I can use this to spot areas that might require more testing. Also I can delete events here, I often use this to bypass input validation on the page and test error handling.</p> <p>The &ldquo;Styles&rdquo; tab shows me the applied CSS. This can help me learn more CSS and also if the CSS seems overly complex I might want to do more cross-browser testing.</p> <h2 id="console-javascript-errors">Console Javascript Errors</h2> <p>The console will show any Javascript errors.</p> <p>Different browsers show different information e.g. some show HTML validation errors.</p> <p>I tend to have the console open all the time so I&rsquo;m aware of any errors because not all errors are displayed on the main web page.</p> <h2 id="recorder">Recorder</h2> <p>The recorder is a new feature which allows recording simple paths through an application and replaying them.</p> <p>These paths can be exported as Puppeteer scripts to make it easier to get started with automating.</p> <p>When the recording is replayed you can also look at the performance report of the execution.</p> <h2 id="network-tab---http-requests">Network Tab - HTTP Requests</h2> <p>The Network tab lets me Observe HTTP messages as they are issued.</p> <p>I can also select the messages to Interrogate and see them in detail.</p> <p>I tend to use the following settings</p> <ul> <li><input checked="" disabled="" type="checkbox"> Preserve Log - so that requests are not cleared when I navigate through the application</li> <li><input checked="" disabled="" type="checkbox"> Disable Cache - this is slightly unnatural because the browser cache is not used but it lets me make sure I don&rsquo;t miss a request, if you test with this option on then it is important to do some testing such that the cache is used so you see if there are any obvious issues involving caching.</li> </ul> <p>Right clicking on a request opens up a new world of options, particularly the ability to:</p> <ul> <li>Save all as HAR with Content - you can save the requests as an HTTP Archive file which can act as a record of your testing and you can view it in a HAR File viewer or HTTP Proxy</li> <li>Copy as &hellip; - copy the request in various formats such that it can be replayed or amended in other tools e.g. copy as cUrl allows pasting a request that triggered a bug into a defect report</li> </ul> <h2 id="sources-tab">Sources Tab</h2> <p>This allows me to read and review all the Source code.</p> <p>Additionally the &ldquo;Snippets&rdquo; view is very useful for writing small Javascript utilities.</p> <p>You can find more information about this in my free Test Automation U course:</p> <p><a href="https://testautomationu.applitools.com/automating-in-the-browser-using-javascript/">https://testautomationu.applitools.com/automating-in-the-browser-using-javascript/</a></p> <h2 id="performance-and-memory">Performance and Memory</h2> <p>I don&rsquo;t use this section very much but you can use this to profile the layout and rendering performance, also capture memory profiles and snapshots of memory.</p> <h2 id="application-storage">Application Storage</h2> <p>I use the Application Storage frequently to observe how the application is storing data e.g. is it using Cookies, Local Storage or Session Storage.</p> <p>I also use to to manipulate the data to clear or amend it to put the application into different states.</p> <p>I also Interrogate the data to see if any confidential information is being stored on the client which might cause a security issue for the user.</p> <h2 id="lighthouse">Lighthouse</h2> <p>Lighthouse is an excellent tool for reporting on performance and potential cross browser issues.</p> <p>Since it runs locally it may not be representative of all your users performance issues.</p> <h2 id="summary">Summary</h2> <p>Becoming familiar with the Dev Tools can help you learn HTML, CSS, HTTP and Javascript. When I&rsquo;m not using the dev tools I often feel like I&rsquo;m testing blind because I have no insight into the internal workings of the application in my browser.</p> <div class="video-container"> <iframe width="560" height="315" src="https://www.youtube.com/embed/cwLxQ68SLqA" allow="autoplay; encrypted-media" allowfullscreen></iframe> </div> <div><p class="center-text"><a href="https://www.youtube.com/watch?v=cwLxQ68SLqA" target="_blank">Watch on YouTube</a></p></div> <p>The video sections are:</p> <ul> <li>00:00 Chrome Dev Tools in Software Testing</li> <li>00:15 The Pulper SUT</li> <li>00:28 The Dev Tools</li> <li>00:51 Element Selector</li> <li>01:23 Mobile View Device Toolbar</li> <li>04:40 Elements Tab DOM View</li> <li>11:44 Console Javascript Errors</li> <li>12:33 Recorder</li> <li>14:23 Network Tab - HTTP Requests</li> <li>20:23 Sources Tab</li> <li>21:57 Performance and Memory</li> <li>22:48 Application Storage</li> <li>24:19 Lighthouse</li> <li>26:28 Summary</li> </ul> <p>If you found this useful. Check out my online course <a href="https://www.eviltester.com/page/onlinetraining/techwebtesting101/">Technical Web Testing 101</a></p>&lt;p&gt; &lt;strong&gt;&lt;a href=&#34;https://www.eviltester.com/ebooktop6&#34;&gt;Read our free ebook&lt;/a&gt; on the Software Testing books you Must Read (and Why)&lt;/strong&gt; &lt;/p&gt;