AutoE
Revolutionizing web application testing
Quick Facts
Context
With all digital markets expanding, there is an exponentially increasing need for testing. Software testing statistics show that quality control is still the number one industry challenge. The evolution of artificial intelligence is pushing manual testing to the side, yet there is still a lot revolutionizing that can be done. Can we push the bar on automation testing to make it available to all?
​
Problem:
How might we create an automation testing tool that is easy to use for a wider demographic of people (more than just quality engineers) than what currently exists in the market?
​
Solution:
Create a desktop application prototype for 2 user flows:
1. Sign up/onboarding application flow
2. Creating a test flow
​
Startup Name: Maeris
​
Startup Team:
Me (Product Designer) + the CEO, CTO, (1) team lead, (3) software engineers, (1) marketing head, (1) product manager, and (3) technical advisors.
​
Prototype:
The Process
Research, Definition, and Ideation
​
Before I was onboarded, there was extensive market research done on this industry, as well as research on competitors, all confidential. On the way to achieving quality at speed, there are two major roadblocks: frequent requirement changes and the lack of time. When it comes to testing, there are (4) types that can be automated as they are repetitive, involve a lot of data entry, and have clear pass/fail results.
​
I researched screenshots of the competitor's key screens to analyze what was successful/a pinch point. I also watched youtube video's of their flows to understand application testing flows.
​
​
​
​
​
​
.png)
.png)
.png)
After understanding the industry for testing automation, we came up with two initial How Might We questions to define the problem and start initial brainstorming. We already had a sound start in the analysis of competitors to release a similar product, which already pre-defined the problem - but our product needed to be more regulated and efficient.
​
1. How might we create an onboarding flow that celebrates our features to separate from competitors?
2. How might we streamline and visualize the testing flow to make it easy for not just quality engineers but all employees of tech companies?
The heavy lifting for my role with AutoE began analyzing existing research to ideate, prototyping, user testing, and iterating.
Group brainstorming sessions involved real time consolidating and organizing ideas onto stickies, and rearranging them to resemble and early flow. This developed into sketches (both hand drawn and virtual) and more fine tuning of the user flows and information architecture. What was interesting was that Miro and FigJam allowed us to jump on many Google Meet calls and discuss concepts, pages, arrangement, and ease of flow live while editing.
​
​
​
​
​
​
_JPG.jpg)
_JPG.jpg)
Digital Screen Sketches
As this was a startup and an agile process - time being of essence - we moved right into user task flows and information architecture, again working real time to rearrange, edit, and iterate. These were the two main flows we came up with to show the initial product and what it could do to venture capitalists:.
Collaborative Early Brainstorm Flow
Information Architecture​
​
Take Flow 1: Sign up flow for AutoE and onboarding of your application for testing
Task Flow 2: Creating a test case scenario for the first time
Prototyping and Iteration​
​
Low Fidelity Mockups
From the user flows and architecture, I moved to wireframes/lo fidelity mockups, adding random color and random graph to signify that these items were design elements I was thinking about. I started with shades of blue, which everyone ended up liking for the branding. A lot of the verbiage was a placeholder for the software engineers to help me fill in.
​
​
​
​
​
​
​
Task Flow 1: Sign up flow for AutoE and onboarding of your application for testing
Task Flow 2: Creating a test case scenario for the first time
​
​
User Test Round 1
We then, over the next few weeks completed rolling user testing, of 5 people. starting with 3. These three were employed in the startup and done first, after which we iterated the prototype to add more detail and clarity for a wider demographic.
​
​
​
The tasks asked of these users were:
-
Sign up.
-
Show confirmation receipt.
-
Create new test on dashboard.
-
Run test.
​
​
​
​
​
​
Once this internal feedback was received, I iterated the prototypes per the most common themes, while bumping up the level of detail to high-fidelity. The hierarchal order of feedback addressed is as follows:
​
Sign up flow: flow was successful and easy to navigate, but needs personality.
1. The sign up screen is boring and does not entice the user to sign up. The placeholder colors were well received though, and became the branding color scheme from positive internal feedback.
Before
After
Creating a test flow: flow was successful and easy to navigate with some glitches.
1. The dashboard needed to be revamped.
The minimize-able taskbar on the left needed to be permanently docked.
How would the graphics expand to new pages and more information? This was resolved by
creating "cards."
The information seems crowded. Increase dashboard space.
​
a
b
c
a
b
c
Before
After
2. The smart step buttons needed their own container. Their function gets muddled in the taskbar
currently.
Before
After
3. The run screen "success" was easily lost. Users were not interested in seeing the script, only the final result.
Before
After
4. The colors were funky. During the test, the users did not like the colors, even though it was stressed that they were simply low-fidelity placeholders. After discussion with the team, we decided to brand AutoE with muted earth tones.
​
​
User Test Round 2
After the first round of iteration, we asked two professionals that we knew in the tech field that worked as UX researcher and UX generalist respectively for feedback. The tasks were the same as round 1, but this round provided very detailed commentary on both the experience and the UI. The feedback and iteration is as follows:
​
​
​
​
​
​
​
​
Sign up flow: easy to navigate but background needs function.
1. The sign up screen background is useless.
2. The user wants to see what they are signing up for. The background space could be used for a compelling visual or series of demonstrative points.
3. The language is vague. Welcome to what? The large bold text and header can be more descriptive to set up the user's expectations, maybe accompanied by a graphic.
​
Before
After
4. The guiding labels on the sign up flow was determined to be confusing. There are two credentials the user needs to enter - their account with AutoE and their account with the application they are looking to test. This needed to be clarified; users thought they were entering the same information. twice.
Creating a test flow: flow was intuitive and successful, except for the first screen.
1. The call to action on the first drag and drop screen is not clear - use hints. When users landed on the smart button drag and drop screen, they were unclear what to do. We needed strong onboarding/first time use hints or video guides. Example for each screen below:
​
After
2. The success and failure toasts are still too small. This is the "destination" of this flow; make the size of those notifications equal to their importance.
​
After
After
3. Add a file folder screen. When this test is run, where does it go? Where is it saved? Thus, the test notebook screen was developed as the end of the flow.
After
4. Pay attention to icons and tooltips. Icons were switched out for more conventional ones, buttons were improved, tool tips were added to icons so users did not have to ask, "What does this mean?"
And finally, see our prototype embedded on our own landing page​.
​
Lessons Learned...
Oh my. So many!
-
Developers are teammates. Become friendly with them, get buy in from the bottom up, and make both of your lives easier. It's a song and dance.
-
Fight for what you believe in, do it, and show it. If you show passion and a mockup, you're more likely to sell your ideas.
-
The importance of user testing. Take initiative to make sure everyone is on the same page and organize those user tests/scripts because their feedback is immensely valuable.
-
Don't get attached to the design. Get attached to the audience - design for them.
-
Uncomfortable situations and uncomfortable white space all force you to grow.
-
Have fun and experiment! It leads to a better end product without even trying. Trust me.