Using first click testing to validate website designs

Blog

What’s the problem? 

The Sutton website team has been testing concepts for updates to the council website. The challenge is deciding which design is the most effective to take into a beta testing phase.

One of the methods we’re using to validate effectiveness is a first-click test. This is an activity that captures where on a web page a user first clicks or taps when completing a task. It’s a useful method to understand if users find a website clear and navigable.

Why is click testing important?

Research shows that when users follow the right path on the first click they achieve task success around 87% of the time. This reduces to 46% success if the first click leads down the wrong path. 

These statistics echo the feedback we’re getting about the current website design. We’re hearing that the current design is unclear and often leads users to the wrong areas. The result is a frustrating experience as users are unable to complete their tasks. With this in mind, new designs for the Sutton website need to be thoroughly click tested to ensure users can successfully find what they need.

How do we conduct a click test?

The image below shows two variations of a homepage for the website. Each has a slightly different set of elements and navigation structure. We want to know which variant users find clearest to navigate.

Side by side comparison of two variations of the homepage
Side by side comparison of two variations of the homepage

To test usability we set users a simple instruction. Indicate where on the interface they would go to renew a parking permit. The image below shows the responses received as a heatmap.

A click heatmap showing where users would go to renew a parking permit
A click heatmap showing where users would go to renew a parking permit

Once we’ve received a significant response, the next step is to analyse the data. 

Here we’re looking at three things to confirm whether a design is clear to navigate:

  • Distribution
  • Time
  • Confidence

Distribution

Did users click or tap in places that might lead to a successful task completion? In the parking permit heatmap above, we see that Variant A generated clicks in areas unrelated to the task. Variant B, however, received a more accurate distribution. An irregular click scatter might signal that a design is causing uncertainty. In this case, the labeling and structure should be reconsidered.

Time

How quickly did users decide on where to click or tap? In the heatmap example above, the average click time for Variant A was 10 seconds. For Variant B it was 7 seconds. A longer decision time may indicate the design of the page could be simplified.

Confidence

How confident did users feel that their click or tap would lead to task completion? To determine this we asked users to place how they felt on a confidence scale. In the parking permit example, users tended to feel more confident in their decisions using Variant B. Lower levels of confidence suggest the labeling or design of the interface is unclear.

Going forward

In this example we are confident that variant B generally outperformed Variant A for usability. This does not mean the job is finished. We now need to test how our shortlist of design variants perform when users complete other tasks, for example where would you go to pay a council tax bill. Depending on what we discover here, we may decide to redesign any areas of concern. In this case, we’d repeat the first-click process outlined above, but this time we’d focus more attention to the problem areas.

When we’re finished, we’ll use the evidence to help us decide which website design to focus on in beta.

Luke Piper

Luke Piper

User Researcher



Weeknote #2 (16 July 2021)

Weeknotes

Here’s what we’ve got for you this week:


Every little automation helps councillor enquiries

Chris, Digital Innovations Lead
Trish, Business Partner

We’re working with the Customer Care teams across Sutton and Kingston to help refine and automate their existing councillor enquiry process.  We know there’s software on the market that would help but we wanted to deliver some improvements while we evaluate products, run procurement and implement a solution.

Currently, the process is very manual.  Enquiries from councillors and members of parliament (MPs) are received by email and manually entered into a spreadsheet tracker, assigned to an officer and sent onwards.  Confirmation emails are sent manually, as are chasers and updates.

Our work will introduce a Google Form to create a consistent structure for all enquiries. We expect this will help us collect the information we need and reduce email back and forths to clarify or get more information.  The form submission is automatically saved to a Google Sheet and extra fields are added depending on context.

Councillors need us to confirm receipt of their enquiry and set a deadline for our response.  Our work will automate this process to make the confirmation more timely for councillors, automatically calculate the deadline, and save the Customer Care team from sending this email manually.  The confirmation email will include an automatically assigned reference number to help everyone involved refer to the enquiry.

The Customer Care team can pick an officer to assign from within the sheet using custom menu and interface items we’ve created.  The sheet will email the officer to let them know they’ve been assigned an enquiry and include data from the relevant fields.  

An example of the custom assignment interface we’ve built.
An example of the custom assignment interface we’ve built.

Each day a script updates the status of enquiries, checks deadline dates and emails a reminder to officers, and updates councillors if the deadline is breached. Once a week, a round up email is sent to assistant directors detailing their directorate or team’s performance.

Our next step is to pilot the automated form and sheet with a small group of willing volunteers.  From the response we’ve had already we think even in this initial iteration it will start to add value and increase efficiency.


We want to find the right answer, not have the right answer

David, Service Designer

The waste delivery team is wrestling with ideas around improvements to our services and how we can make sure we make the right improvements. It can be tempting to think that we know what improvements to make but how do we know that we know?

We found Erika Hall’s 2018 talk “Design Research Done Right” very interesting. Erika talks about how much we (all) like to be “right” and how that bias can impact our design research negatively.

In design research we all want to find the right answer but we need to remember that we (should) want to find the right answer, not have the right answer. Most of our professional incentives reward having the right answer and this one reason why design research is tricky.  If we aren’t asking the right question, we may get an answer that gets as an immediate “reward” (praise, promotion, approval for a budget or project request, momentum on a stuck team) but at some larger future cost.

As Erika puts it, “answers have a very short shelf-life”. The world changes and if our answers don’t change with it, they become wrong.Too often those of us in design & technology treat “research” as a stage of our work. We did the research, we got the answer – and that answer turns into an assumption over time. The problem with that is that relying too much on our assumptions introduces risk to our work. Anyone who has worked on a few projects will be able to think of “facts” they got from research that didn’t quite stand up to the real world.  

Erika encourages us to live in the uncertainty of not knowing, to allow ourselves to be uncomfortable with this and to keep asking questions. 

There is so much bad design in the world because people were more interested in defending their answer throughout the process than really asking questions:

  • Is this something people need?
  • Do we have the resources to do it right?
  • Is designing something like this and solving this need going to help us achieve our goals?

Design and User Research leads to evidence based decisions and helps us overcome some of our many cognitive biases. 

There are many great insights and ideas in the talk and if any of these ideas appeal to you:  

  • the need to incentive teams who deliver, not individuals who have answers; 
  • exploring the difference between collaboration and consensus and the need to embrace conflict; 
  • how to influence decision makers when we know data does not change minds; 
  • why even great design teams produce bad design (see Apple & iTunes)!; 
  • the value of a good question to help you make a better bet about user behaviour

…then it’s a good use of 45 mins of your time.

One takeaway is that the adoption of a goal driven and skeptical mindset is a great starting point for design success. 

This rings true to us.


Things we’ve read lately

Pamela’s been reading about Using persona profiles to test accessibility. This is a really interesting concept that you build a profile on a google chromebook and you can then test designs through the lens of a user with accessibility needs. This will then give you insight in your prototype design phase, to ensure it can work for all users.

David has been reading about the challenges of building complexity on a low-code platform as part of his work to rebuild some of the councils’ waste service transactions.

John Paul has been reading how GOV.UK is approaching accounts and what that might mean for service delivery.  This is part of a piece of work we’re doing to consider how accounts are used on council websites.


Weeknote #1 (2 July 2021)

Weeknotes

Hello world!

Here at Kingston and Sutton’s shared digital service we’ve set ourselves an ambition to talk more about the work we’re doing publically. You know: make things open, it makes things better. And in local government there are hundreds of organisations we can learn from and share with.

Inspired by the weeknotes from Digital Dorset, MHCLG’s Local Digital team, Lingjing Yin’s new weeknotes from Greenwich, and many others we’ve decided to give it a ago. To start we intend to publish a fortnight-note with contributions from different teams each time.

Here’s what we’ve got for you this week:


Trying to improve a service while migrating platform adds even more complexity

Paul, Head of Design

This week I’ve been reflecting on how important it is to ensure we understand the level of quality we want to achieve from a piece of work and what is primarily driving us to achieve it. 

One of the projects we’re working on has been getting into difficulty recently. The main issue has been managing the project’s multiple stakeholders and their expectations. Ultimately the project seems to have become a bit confused about what it is trying to deliver. There are a few key drivers underpinning the project:

  • Move functionality away from a legacy platform
  • Transform service delivery
  • Achieve savings

The key take away for me is that the above have very different levels of quality and effort. Lifting and shifting functionality from one platform to another requires careful planning and analysis to ensure a like for like transfer but should essentially be quite straightforward and easy to achieve.

Transforming service delivery on the other hand, often involves solving complex problems. To achieve transformation often requires changes to policy, working practices or sophisticated logic creation, which all takes time.

On reflection I think the problem the project is facing has been caused, in part, because it is trying to deliver on all these drivers at the same time. This makes it very difficult to manage the different expectations around pace and complexity.

So whilst this week has been challenging, it has proved incredibly valuable in thinking about how we might reframe our delivery into more logical chunks of work whose quality expectations can be more easily articulated. 

There’s still work to be done but I think we’ll be much better able to articulate the work of sequencing, why we’re opting for that sequence and what will be achieved at the end of it.


Getting the word out about digital jobs is difficult

Ellie, Senior Business Manager

Over the past few weeks, we’ve been running a recruitment campaign to expand our Digital and IT team.

We had a large number of roles to fill, so we were keen to use the opportunity to increase diversity in the team and be more inclusive in our recruitment campaign. We therefore reached out to a number of sites with specific audiences – those focussed on women in tech (Ada’s List), those with disabilities (Evenbreak) and also specialists in the BAME community (UKBlackTech). We were also eager to utilise previously proven channels such as LinkedIn and also our own social networks. 

This week we ran a retro and here are some of the reflections:

Went well:

  • having budget to promote the roles and enable us to work with a broader range of agencies
  • receiving a lot of applications overall
  • raising our visibility and reputation in the marketplace
  • increased traffic through our newly launched Twitter profile and Blog site.

Challenges we faced:

  • on-boarding niche recruitment sites, as they don’t always have documents in place to meet our business needs
  • high competition from other employers for a small number of skilled candidates, often with shorter application processes and offering higher salaries
  • difficulty calculating the return on investment for our spend on adverts and where to focus our efforts – as third party site conversions (eg user clicking ‘Apply’ button) did not correlate with number of actual submitted applications on our recruitment site
  • the need to include appropriate balance of keywords throughout ad title, tags and content – to ensure they would be discoverable, appear naturally in search results and target relevant job seekers.

Having uncovered some of the main issues in our current processes, we are now working hard to review the lessons learnt and see where we can evolve our practices to achieve optimum results in future campaigns. I will look to share these insights here in the coming weeks.


We need to give contractors a good induction

Pamela, Head of Research

At Kingston and Sutton we rely on contractors to supplement the skills and experience in our digital teams. So this week we’ve been thinking about how we induct them, to ensure they feel part of the team.

We’ve written some points to describe the experience we’d like our contractors have:

  • we should tell the team we’re hiring extra people, so your arrival isn’t a surprise
  • we should welcome you into the team in the same way as everyone else
  • we don’t need to tell everyone you’re a contractor
  • we should assure you that it’s ok not to hit the ground running, because you need to learn about the work first
  • we should be open and transparent about previous work and explain the strategy
  • we should be clear about our ways of working, expectations and standards, as all organisations differ.

One change we’re going to try is buddying contractors with a staff member in a similar role. The staff member can support and advise on our ways of working, preferred approaches and artefacts. This will ensure we’re getting what we need and our staff are able to learn from the contractor’s experience.


Things we’ve read lately

Tom started with us a few weeks ago and has been reading advice on new jobs from:

And about Local Gov Drupal’s Sprint #3 and their feature design process (access request needed).

Jon‘s been reading Managing for Happiness. The author, Jurgen Appelo, gave the Agile2016 keynote on Managing for Happiness. The book captures much of the presentation and is full of concrete games, tools, and practices for all workers.

Our website team have been reading up on GOV.UK Accounts to learn about their approach and what it might mean for service delivery.

Embedding digital across Kingston and Sutton councils

Digital Transformation

Steve O’Connor our Chief Digital and Information Officer sets out how we have been transforming our digital approach for the Royal Borough of Kingston upon Thames and the London Borough of Sutton.

When I first joined Kingston Council in early 2020, placing user-centred design at the heart of our work was already on my mind.

More than a year later, and in spite of the challenges that the pandemic has thrown at us, it’s exciting to see how far we’ve come in achieving our goal. 

Working more in the open was also a key consideration. So in that spirit, I’m going to tell you how we’ve established digital culture within Kingston and Sutton councils, and what’s coming up next.

Building digital capability within the team

At the core of what we’ve been doing is building a digital culture that puts the user at the heart of everything we do.

When COVID-19 hit this time last year, we went immediately into response mode. Our resources were rapidly funneled towards supporting the challenges posed by COVID. Our teams were building digital solutions to support the urgent response of both councils – from requesting help, to registering as a volunteer, through to enabling the delivery of food and support.  

As we emerged from the first lockdown, we applied what we had learnt through the COVID-19 response work and kicked off Digital Transformers, a program to reflect on where we were and where we would like to get to. Taking a service design approach ensured our focus was firmly fixed on the digital delivery we offer to both councils. We reviewed our end-to-end service from the perspective of our users (including service areas such as Housing and Health and Social Care), our residents and local businesses, and also our staff who are delivering the service. This 360 perspective helped us to identify opportunities in terms of organisational structures, working practices, internal processes and overall governance.

A key consideration was the capability and capacity of the team needed to design and deliver user-centred services. We were very fortunate to begin the process with a talented pool of people, who needed a little support to develop into more specialist digital roles. To do this we organised mentoring and coaching for people to move into digital and user-centred design roles including delivery managers, service designers and user researchers. As a result, we now have skills and capabilities that we just didn’t have this time last year. 

Establishing a digital culture

Previously, services were built in a very functional way. We would work with a part of the council on the build and the end-user would be consulted somewhere during the process, but we didn’t go as far as conducting proper user research or user testing.

Today, we’ve not only invested in new digital roles and capabilities, but we’ve also built our processes to be more user driven. It’s been quite a big cultural shift, but it has meant that we’re now thinking carefully about how we design our services. Working across the full spectrum – from improving everyday council services, making them easier to do online, to fundamental work such as social care and the role of digital within that, to more innovative work using sensors and IoT (Internet of Things). This is work that really makes a difference to people’s lives. You can very quickly see the impact of your work.

We have also put digital at the heart of transformation at both Kingston and Sutton councils. Previously digital was seen as a separate function, whereas now digital is considered a key component of all transformation work. We have embedded a new agile process into our ways of working, from pre-discovery through to live. Whereas before a service might come to us with a solution, we now work with the service team to articulate the problem they are trying to solve. Through multi-disciplinary teams we dedicate time for Discovery to properly understand the needs of the user, the goals of the service, and the current systems. We then undertake co-design with users and stakeholders to ideate solutions and develop a design hypothesis that we can be tested throughout Alpha. At the end of Alpha, we will have a fully thought through ‘service vision’ which can be taken forward.

What’s coming up

It’s an exciting time for both councils as we continue to build capability in the existing team as well as grow the team further. 

There are so many opportunities for digital to make a difference to people’s lives. We’re about to scale up rapidly as we work on a number of projects across the whole spectrum of council services, but more on that to come.

As part of our new dedication to working in the open, you can look forward to reading blogs from us about:

  • The digital projects and services we’re working on
  • Our process, methodologies and workflow
  • Opportunities to join our growing digital team

We’ll also be talking about the work we’re doing on our technology, such as moving to the cloud and establishing new capabilities around data.

Follow our progress

Local government has a brilliant culture of sharing, so we want to start working more openly and to share what we’re doing with the community. We’re also keen to make it a two-way conversation, where we can learn from others. This site is the first step for us, and will be the new home of our blog posts, weeknotes and, hopefully one day soon, recordings of show and tells and more.

We’ve also recently launched a Twitter profile for the digital team here at Kingston and Sutton. It’s early days but you can follow us at @ksdigital01.

Steve O'Connor

Steve O’Connor

Chief Digital and Information Officer