Experiment Idea: Iterate on copy in "Novice or Experienced" page in onboarding board experiment.
Problem
Launching the new onboarding experiment using an issue board, we added an extra step where users can tell us about their level of experience with DevOps. They can choose between Novice and Experienced. Data tells us that 76% of people in the experiment declared themselves as Novice. That said this screen presents some copy that leaves room for interpretation and perhaps bias users towards the novice option. Could the data be skewed?
Design | Data |
---|---|
Novice:
- Copy: "I'm not familiar with the basics of project management and DevOps"
- CTA: "Show me everything" - This call to action might be nudging users towards this option.
Experienced:
- Copy:"I'm familiar with the basics of project management and DevOps"
- CTA: "Show me more advanced stuff"
Proposed Solution
Iterate card copy and see how this impacts data:
Novice:
- Copy: "I'm not familiar with the basics of DevOps"
- CTA: "Show me the basics"
Experienced:
- Copy:"I'm familiar with the basics of DevOps"
- CTA: "Show me advanced features"
Experiment rollout & tracking
We should make this change to the existing onboarding experiment experiment_id
= 1
and simply mark the date of the change and ensure we have a screenshot of the old vs new language. We can then utilize our existing reporting to see how the answer to this question changes after the new language is in production.