![Physical Product Experiment [PPE #5] – Product Testing](https://buildpassiveincomeandwealth.com/wp-content/uploads/2017/09/cropped-BPIWHeader6b.png)
If you’re following along, this is the fifth installment of my Physical Product Experiment, a step-by-step blog post series where I share a behind-the-scenes look at each stage in the process of building my first physical product: a productivity calendar and workbook. It’s been an exhilarating ride and a really amazing learning process. For a full recap, you can check out the previous installments here:
- In Physical Product Experiment [PPE #1], I introduce what the experiment is all about, and why I chose to explore physical products.
- In Physical Product Experiment [PPE #2], I discuss two of the most important components to the experiment’s success: validation and feedback.
- In Physical Product Experiment [PPE #3], I get into the nitty-gritty with prototypes!
- In Physical Product Experiment [PPE #4], I share my thoughts on the sometimes challenging manufacturing and shipping process.
Last month, I mentioned that we sent out the prototype testing packages, with medium fidelity versions of the physical product (wall calendar, workbook, and stickers), to our twenty-one beta students. I’m happy to say that product testing is in full swing!
If you recall, in this first iteration of the productivity calendar/workbook combo, the students are testing it out in their pursuit of achieving one specific goal they’ve expressed passion in pursuing: writing the draft of a book in ninety days—all the while providing feedback along the way!
In addition to the physical product testing packages, the beta students have access to the SPI Goal Quest Slack Community, where they are actively sharing their essential feedback about the process. We’re already learning so much (thanks, students!), which we’ll get into in more depth later in this post. At this point in the process, we’ve already learned so much thanks to the beta students, which will enable us to further fine-tune and make changes to the product so we can eventually deliver the best possible calendar/workbook to the public.
But before we get into the lessons we’ve already learned, I’d like to share a couple of behind-the-scenes snapshots of the beta testing packages we sent out.
Fun, right? Thanks to my team for the hard work at putting those packages together and shipping them out to the students! There’s something special about the fact that it’s a physical, tangible product (as opposed to a digital one), that just makes it seem more real—for me and the students.
Physical Products vs. Digital Products
During this experiment, it’s been clear that testing actual physical representations of the calendar and workbook (medium-fidelity versions of the final product) has been a good choice.
Interacting with something that you can feel and touch is a more engaging experience. As students play around with the product, they have the ability to take note of how the material feels in their hands, if the stickers adhere in a way that’s expected, and so on. It’s a more powerful experience compared to a digital experience with the same type of product.
In fact, before we had even sent out the testing packages, we received feedback from a number of excited beta students eager to open their testing packages to get started.
Engaging with a physical product is a different experience than engaging with a digital product. With a physical product, there’s an extra, sensory level of experience that can’t be replicated with digital documents.
But with physical products there’s also an aspect that’s unknown—not just because I’m new to the world of physical products, but because there’s a lot that can potentially go wrong, from things being misprinted to products getting damaged during the shipping process. When things like that happen, changes are much more difficult to make on a physical product because the damages may be permanent (e.g., If a calendar was delivered torn and damaged, we couldn’t just send some tape over; we’d of course need to send a brand new one).
It’s my job, and my team’s job, to make sure we don’t get stuck going down a particular path simply because we’ve already made decisions about the look, feel, and design of the product. We have to be flexible, especially considering we’re getting feedback from the beta testers. And with a physical product, that takes more time and patience.
Product Testing Begins!
As I mentioned before, our beta testers have started testing the product (they’re about two weeks into the ninety-day process). While they test, we’re working with the manufacturer to determine the final product materials and to produce samples of the calendars and workbooks that more closely resemble the final version.
Our first hiccup in product testing came as a result of unclear instructions we sent to our beta testers. The instructions were related to the Slack community, but we realized after we sent out the packages (which also included a welcome letter from me) the instructions were actually intended to be read online, so there were calls-to-action like “click here” on a physical piece of paper. Oops!
To fix this, we quickly sent our beta testers the links they needed to join the Slack community. It was a small hiccup, but it highlights why we run beta tests. When you test, that’s when you iron out all of the little issues so that when you launch publicly, you don’t run into any problems.
It might seem obvious that this hiccup shouldn’t have happened, but when you are deep in the trenches of a project and there are several moving parts, it’s actually very easy to miss little things like this.
This is why you test and test and test again.
I can’t tell you how many times I’ve seen episodes of Shark Tank in which people come in with a physical product to sell, and they go on and on about how they built the product, how much they spent on the product, but clearly they didn’t test the product based on the reactions from the panelists.
Going through this now with my physical product, I totally understand how the excitement for the product you’re building and hoping to put out into the world can easily take you away from the nitty gritty details. But that’s why you have a team. That’s why you have a beta-testing group to help cover that part of the process.
As the product owner, I need to remind my beta testers that they didn’t receive final versions of the product—and so there may be rough spots that still need to be polished. At the same time, I need to help them understand that there is value, even with the “beta” version of the product.
One other hiccup we ran into was the die-cutting on the stickers was not perfectly aligned, so the stickers themselves were a bit off. That’s a manufacturing-related bump we didn’t expect, but thanks to the feedback from the beta group, we will resolve the issue before the final launch.
SPI Goal Quest Slack Community Update!
Mikky, our SPI Goal Quest Slack community manager, has three updates to share about the goings-on in our Slack community. Here’s what she has to say:
- Accountability and motivation. “So far, I’ve seen how powerful a community can be. Surrounding yourself with a community of people working toward the same goal boosts accountability and motivation.”
- Sharing unique perspectives. “The community has been very active. Pat chimes in regularly and the beta testers have been frequently sharing their wins, struggles, and advice with each other, providing unique perspectives and fast, relevant feedback.”
- More relevant writing prompts. “One integral piece of feedback we’ve heard from the Slack community is regarding the daily writing prompts that Team Flynn came up with beforehand. The community would rather see daily writing prompts that are directly related to the final goal and parallel with the chapters, as opposed to some of the more creative/less focused that we had been sharing (e.g., ‘Write about the typical day of your avatar’). We’re already making adjustments to make the writing prompts more relevant to the final goal.”
Thanks for the update, Mikky!
Listening and Paying Attention
Overall, I’m happy to say that the feedback from the beta group has been very positive. There’s a consensus about how the workbooks helps to focus the user’s mindset toward the goal of writing. Write the First Draft of Your Book is the ninety-day goal they are seeking to accomplish in this first go-around, so that focus is what we want!
The overall calendar/workbook product is about the process of starting and finishing a goal in ninety days so that we can determine whether or not this is an actual process that may work with other goals down the road. Our big idea is to have other goals available for people to choose from, a library of goals to seek and accomplish. That would be really cool!
Our ongoing job is to make sure we listen to our beta testers, take in their feedback, and make the necessary changes. We also need to pay careful attention to the whole process, and ask questions of our beta testers (and of ourselves) to make sure we’re looking at this from every angle. We’ve already learned so much, which is inspiring and motivating.
Failure is an Option
Internally, we’ve been talking about marketing the product. We’re really excited with the branding so far, and can’t wait to get the final versions ready for later this year! Then, the idea is to add different calendar/workbook products targeting specific goals ready for early next year, and build that library as long as it’s going well.
But, like with anything, it may not go well. It may fail. And failure is okay. Failure teaches you something. Failure gives you the opportunity to improve it for next time, or try something different. And we can’t know the outcome until we try, right?
So we’re moving forward!
And, to finish it off, here are a couple of images from Kristie Winget on our beta testing team!