Designing Effective Qual Research Protocols for Measuring AI Trust | Dovetail Champions

Designing Effective Qual Research Protocols for Measuring AI Trust

Hi folks! A big question I've been wondering about: How can we design qual research protocols that effectively measure user trust and acceptance of AI functionalities while accounting for the varied AI literacy of participants? AI tool usage is still new among mainstream audiences so as familiarity and attitudes shift, how do you avoid locking stakeholders into assumptions about user behavior that may hold true now but may shift soon?

👀1

6 comments

Jocelyn S.
·
·
Frame it by how attitudes towards AI are shifting overtime. If you don't have data for your users overtime, you could borrow from broader perception studies for this part. https://www.pewresearch.org/short-reads/2023/11/21/what-the-data-says-about-americans-views-of-artificial-intelligence/sr_23-11-21_ai-roundup_1-png/
Sean S.
·
·
Jocelyn S. I don't have data from my own users OVER TIME. I guess that's what I am looking for: LONGITUDINAL studies about how perceptions are CHANGING (and when/if some of them are staying the same). Those Pew data points above are a great case in point: they show several interesting snapshot temperature-checks from a moment in 2023 - first knee-jerk reactions... but we lack perspective about how perceptions are evolving
Sean S.
·
·
(My POV is: we just don't have this to draw on yet, but it's important to keep on top of new studies in this space)
Jocelyn S.
·
·
I guess I was answering just the last part of your question.
Nadyne R.
·
·
Can you say something more about why you're looking to measure something using qualitative research? When I've been interested in a fast-moving landscape before, I've run my own tracking quant survey for my target audience with a set of questions to measure the things we cared about. For example, I was working for a healthcare company at the beginning of covid. It was clear to me that people's attitudes about telehealth were going to shift quickly, which was important to our company and our business. I did a monthly survey for the first 12 months, then moved to quarterly. I left that company a couple of years ago, but understand that they're still running that survey twice per year.
👍1
Sean S.
·
·
Nadyne R. That's fair, quant measures might be a better fit. Most of the results that spur these "but will this change?" question come from academic research papers (my own company doesn't have its own major AI products in the marketplace at this point) so I'm not measuring this myself but definitely need to start thinking about how we will monitor reception of, and attitudes around, AI products down the line.