Physical Product Experiment [PPE #5] – Product Testing

If you’re following along, this is the fifth installment of my Physical Product Experiment, a step-by-step blog post series where I share a behind-the-scenes look at each stage in the process of building my first physical product: a productivity calendar and workbook. It’s been an exhilarating ride and a really amazing learning process. For a full recap, you can check out the previous installments here:

  • In Physical Product Experiment [PPE #1], I introduce what the experiment is all about, and why I chose to explore physical products.
  • In Physical Product Experiment [PPE #2], I discuss two of the most important components to the experiment’s success: validation and feedback.
  • In Physical Product Experiment [PPE #3], I get into the nitty-gritty with prototypes!
  • In Physical Product Experiment [PPE #4], I share my thoughts on the sometimes challenging manufacturing and shipping process.

Last month, I mentioned that we sent out the prototype testing packages, with medium fidelity versions of the physical product (wall calendar, workbook, and stickers), to our twenty-one beta students. I’m happy to say that product testing is in full swing!

If you recall, in this first iteration of the productivity calendar/workbook combo, the students are testing it out in their pursuit of achieving one specific goal they’ve expressed passion in pursuing: writing the draft of a book in ninety days—all the while providing feedback along the way!

In addition to the physical product testing packages, the beta students have access to the SPI Goal Quest Slack Community, where they are actively sharing their essential feedback about the process. We’re already learning so much (thanks, students!), which we’ll get into in more depth later in this post. At this point in the process, we’ve already learned so much thanks to the beta students, which will enable us to further fine-tune and make changes to the product so we can eventually deliver the best possible calendar/workbook to the public.

But before we get into the lessons we’ve already learned, I’d like to share a couple of behind-the-scenes snapshots of the beta testing packages we sent out.

Fun, right? Thanks to my team for the hard work at putting those packages together and shipping them out to the students! There’s something special about the fact that it’s a physical, tangible product (as opposed to a digital one), that just makes it seem more real—for me and the students.

Physical Products vs. Digital Products

During this experiment, it’s been clear that testing actual physical representations of the calendar and workbook (medium-fidelity versions of the final product) has been a good choice.

Interacting with something that you can feel and touch is a more engaging experience. As students play around with the product, they have the ability to take note of how the material feels in their hands, if the stickers adhere in a way that’s expected, and so on. It’s a more powerful experience compared to a digital experience with the same type of product.

In fact, before we had even sent out the testing packages, we received feedback from a number of excited beta students eager to open their testing packages to get started.

Engaging with a physical product is a different experience than engaging with a digital product. With a physical product, there’s an extra, sensory level of experience that can’t be replicated with digital documents.

But with physical products there’s also an aspect that’s unknown—not just because I’m new to the world of physical products, but because there’s a lot that can potentially go wrong, from things being misprinted to products getting damaged during the shipping process. When things like that happen, changes are much more difficult to make on a physical product because the damages may be permanent (e.g., If a calendar was delivered torn and damaged, we couldn’t just send some tape over; we’d of course need to send a brand new one).

It’s my job, and my team’s job, to make sure we don’t get stuck going down a particular path simply because we’ve already made decisions about the look, feel, and design of the product. We have to be flexible, especially considering we’re getting feedback from the beta testers. And with a physical product, that takes more time and patience.

Product Testing Begins!

As I mentioned before, our beta testers have started testing the product (they’re about two weeks into the ninety-day process). While they test, we’re working with the manufacturer to determine the final product materials and to produce samples of the calendars and workbooks that more closely resemble the final version.

Our first hiccup in product testing came as a result of unclear instructions we sent to our beta testers. The instructions were related to the Slack community, but we realized after we sent out the packages (which also included a welcome letter from me) the instructions were actually intended to be read online, so there were calls-to-action like “click here” on a physical piece of paper. Oops!

To fix this, we quickly sent our beta testers the links they needed to join the Slack community. It was a small hiccup, but it highlights why we run beta tests. When you test, that’s when you iron out all of the little issues so that when you launch publicly, you don’t run into any problems.

It might seem obvious that this hiccup shouldn’t have happened, but when you are deep in the trenches of a project and there are several moving parts, it’s actually very easy to miss little things like this.

This is why you test and test and test again.

I can’t tell you how many times I’ve seen episodes of Shark Tank in which people come in with a physical product to sell, and they go on and on about how they built the product, how much they spent on the product, but clearly they didn’t test the product based on the reactions from the panelists.

Going through this now with my physical product, I totally understand how the excitement for the product you’re building and hoping to put out into the world can easily take you away from the nitty gritty details. But that’s why you have a team. That’s why you have a beta-testing group to help cover that part of the process.

As the product owner, I need to remind my beta testers that they didn’t receive final versions of the product—and so there may be rough spots that still need to be polished. At the same time, I need to help them understand that there is value, even with the “beta” version of the product.

One other hiccup we ran into was the die-cutting on the stickers was not perfectly aligned, so the stickers themselves were a bit off. That’s a manufacturing-related bump we didn’t expect, but thanks to the feedback from the beta group, we will resolve the issue before the final launch.

SPI Goal Quest Slack Community Update!

Mikky, our SPI Goal Quest Slack community manager, has three updates to share about the goings-on in our Slack community. Here’s what she has to say:

  1. Accountability and motivation. “So far, I’ve seen how powerful a community can be. Surrounding yourself with a community of people working toward the same goal boosts accountability and motivation.”
  2. Sharing unique perspectives. “The community has been very active. Pat chimes in regularly and the beta testers have been frequently sharing their wins, struggles, and advice with each other, providing unique perspectives and fast, relevant feedback.”
  3. More relevant writing prompts. “One integral piece of feedback we’ve heard from the Slack community is regarding the daily writing prompts that Team Flynn came up with beforehand. The community would rather see daily writing prompts that are directly related to the final goal and parallel with the chapters, as opposed to some of the more creative/less focused that we had been sharing (e.g., ‘Write about the typical day of your avatar’). We’re already making adjustments to make the writing prompts more relevant to the final goal.”

Thanks for the update, Mikky!

Listening and Paying Attention

Overall, I’m happy to say that the feedback from the beta group has been very positive. There’s a consensus about how the workbooks helps to focus the user’s mindset toward the goal of writing. Write the First Draft of Your Book is the ninety-day goal they are seeking to accomplish in this first go-around, so that focus is what we want!

The overall calendar/workbook product is about the process of starting and finishing a goal in ninety days so that we can determine whether or not this is an actual process that may work with other goals down the road. Our big idea is to have other goals available for people to choose from, a library of goals to seek and accomplish. That would be really cool!

Our ongoing job is to make sure we listen to our beta testers, take in their feedback, and make the necessary changes. We also need to pay careful attention to the whole process, and ask questions of our beta testers (and of ourselves) to make sure we’re looking at this from every angle. We’ve already learned so much, which is inspiring and motivating.

Failure is an Option

Internally, we’ve been talking about marketing the product. We’re really excited with the branding so far, and can’t wait to get the final versions ready for later this year! Then, the idea is to add different calendar/workbook products targeting specific goals ready for early next year, and build that library as long as it’s going well.

But, like with anything, it may not go well. It may fail. And failure is okay. Failure teaches you something. Failure gives you the opportunity to improve it for next time, or try something different. And we can’t know the outcome until we try, right?

So we’re moving forward!

And, to finish it off, here are a couple of images from Kristie Winget on our beta testing team!

Physical Product Experiment [PPE #4] Manufacturing & Shipping

It’s the fourth installment of the Physical Product Experiment! In case you missed the previous installments, make sure to check those out before you read on. You can find those posts here:

  • In Physical Product Experiment [PPE #1], I introduce what the experiment is all about, and why I chose to explore physical products.
  • In Physical Product Experiment [PPE #2], I discuss two of the most important components to the experiment’s success: validation and feedback.
  • In Physical Product Experiment [PPE #3], I get into the nitty-gritty with prototypes!

To jog your memory, the Physical Product Experiment is a brand new venture for me. It’s super exciting, but it’s also something that could potentially fail (like with anything, I guess). But that’s why I am taking my time with it. I want to do it right, make sure I thoroughly test out the idea before releasing it into the public. I want it to be a success. I’ve never created a physical product before, but it’s something I’ve always wanted to do.

Initially, the physical product I envisioned was a productivity calendar. But, as time passed and I did more research, that concept morphed into something different: a physical productivity calendar paired with a workbook, which interact with each other. And we’re not talking about a blank workbook here. The idea is that the calendar and workbook would be created with the goal of helping people achieve their specific, targeted goals.

For instance, during our validation and feedback part of the experiment (which is happening now; more on that below!), twenty-plus amazing volunteers signed up to test out the physical calendar and workbook prototype in their pursuit of achieving one specific goal they’ve expressed passion in pursuing: writing the draft of a book in ninety days. In exchange, the volunteers would share their feedback about the experience.

Down the road, the plan is to have a wide spectrum of 90-day goals for people to choose from, each goal with a specific roadmap for the user to follow. Since they are 90-day goals, this leaves time for people to achieve several goals throughout the year using the same productivity calendar and workbook!

But that’s down the road. One more immediate obstacle we’re facing in this experiment is the challenge of actually creating, manufacturing, and distributing a physical product. It’s a lot more complex and difficult than you’d think! Richie Norton, founder of Prouduct, reminded me of this in a recent episode of the Smart Passive Income Podcast (a must-listen episode if you’re thinking about creating a physical product).

Why We Beta Test

As I explained in Physical Product Experiment [PPE #3], the twenty-one volunteer beta testers will be receiving prototype calendar and workbooks soon. They’ve been patiently (thank you!) awaiting those packages as we iron out the kinks, but we’re happy to say now that we’ve started sending out the prototypes for the first round of beta testing!

Beta testing is really important for obvious reasons (see my book, Will It Fly?, for more reasons why). For a physical product, beta testing is even more crucial because we need to not only understand how people use the product, but also what people think about the overall design of the product and how it feels in their hands. The tactile experience users have is an aspect of beta testing that can’t be ignored.

So, we’ve been taking this physical product experiment very slowly. We want to make sure we do it right. There are a lot of costs involved, as you might imagine, especially when it comes to manufacturing. There are also still many unknowns in terms of how many of these productivity calendar-workbooks we’ll sell, which almost makes it feel like gambling in a way—when you have to pick a specific number of products to manufacture before you make your order. It feels risky. But that’s why we beta test—to get the best possible idea of how the product will perform once released into the wild.

Process Is Key

There are still a lot of things up in the air, but we’ve made progress. By the time this blog post goes live, the twenty-one volunteer beta testers will have already received a box that includes digitally-printed versions of the final calendar, workbook, and sticker set designs. The digitally-printed versions of the products are approximate representations of the final design, so they won’t be the final final in terms of product material, paper, and print quality.

The reason for this is we needed to meet our deadlines. In the research phase for this physical product experiment, we spoke with numerous experts who suggested we explore options, price points, and other details related to the shipping of the physical product. This ended up being overwhelming and time-consuming. So, because of this, our timeline shrunk, and we needed to move forward with digitally-printed versions in order to get the products into the hands of the beta testers so they can start testing the process.

This will enable the beta testers to kick off their 90-day draft writing project and provide feedback on the process of the product itself. The process is the most important, after all. The process is the guts of the product, what will make or break the success of the goal seeker, and we need to make sure that it’s as finely tuned as possible before we go public.

The final product materials, in terms of feel, is also important. But the more extensive feedback loop will take place in the process testing from the beta testers. So, at this point, even though we’d love to get feedback on the product materials, we’re just not there yet, and we needed to get the product into the hands of the beta testers in order to meet our November promotion and December shipping timeline goals.

Why December?

During December, goal planning is top-of-mind for a lot of people who want to kick off the new year right. 2018 will be here before we know it!

Collecting Feedback

In addition to sending the products to our beta testers, we’ll also share it in the Slack Community. The Slack Community is going to be really important, because this is how we’re going to collect feedback during the 90-day process. We chose Slack because it differs from Facebook, for example, in that it provides a bit more customization and offers more privacy.

Plus, Slack allows for a less “noisy” atmosphere. Specific channels will be created for specific conversations, such as a #feedback channel. The complete list of channels will include:

The Cost Factor

Costs are going to be an interesting factor down the road. Placing a bulk order upfront is the plan, as I mentioned before, but how will we really know how many to order? The plan is to pre-sell. By pre-selling, we’ll get a more accurate understanding of the demand. It’s a way to validate the interest (see that validation thing again; it’s important!).

We also have to consider that there is potentially a four-to-six-week manufacturing and shipping process (thankfully, I have Richie and his Prouduct team helping manage the manufacturing process). And we’ll be working with another third-party company to help focus our efforts related to shipping.

Another cost factor (another one, yes) is, since there are multiple components to this product (calendar, workbook, stickers), additional steps are necessary, which translates to additional costs. We want to create an amazing product that’s reasonably priced, but to do that properly takes time and patience to ensure we have the steps in place—from shipping to manufacturing to materials—to get it done effectively and as cost-efficiently as possible.

I’ll share more specifics about the costs in the next installment!

An Epic Quest

As I explained in PPE #3, Atlas Press is the name for my new business unit within SPI that will produce a line of products, and we want the products themselves to have names that tie into the “atlas” theme (collection of maps, tables, and charts). So we chose a brand name that speaks to the journey the users will go through during the process:

Quest.

The calendar will be branded as Quest Maps. The workbook will be branded as Quest Books. The overall product name will then be called a Goal Quest.

The plan is to have different colors for different groups of goals. For example, there might be a goal related to fitness, which might be a certain-colored book, and a goal related to personal work-life balance, which would be marked as a different color. I really like the idea of this goal quest and having it be like an expedition, something that users will trek through, and that will take work, but by using the guides as your map, you can help find your way to the goal.

Let’s hear a quick take on the design strategy we took from Team Flynn designer Phil:

The book is meant to tell a story while guiding people on their own journey in finding their way through writing their first book, and inspire them along the way. Right from the cover, we’re depicting the journey that this will be, using the topographic visual asset to imply discovery and exploration.

Knowing this could be a challenging endeavor for some, we wanted to make it easy on our readers to make their way through the pages, and always know where they stood in the process. So we designed very deliberate spreads to showcase progress with large typography denoting the milestones in sequence, and sprinkling words of wisdom to start each new chapter. A little positive reinforcement can go a long way from those who have achieved great things.

Additionally, throughout the story there are little breaks in the action where we ask our readers to pick up their pens and engage with the pages of the book. Here, again, we want our audience to know exactly when we are asking them to participate, so we designed specific call out box and workbox page styles to “guide” our users through their portion of the story. Throughout those sections in the workbook, we are using simple ruled lines, and plenty of them (in case someone has a lot to capture).

In the end we want these pages to be filled with the creative thoughts, words and ideas of our readers, and we hope the design of the new book allows each person to do just that!

All the Little Details and Next Steps

So, things are moving forward, just a little bit slower than we anticipated because of all the details about manufacturing and shipping and materials we had to nail down with the team over at Prouduct. But, that’s to be expected. We’re still learning as we’re putting the plan together. Working with Prouduct, a team with vast experience in these matters, makes a world of difference in helping me understand nuances I may not have seen or understood otherwise.

For instance, what’s the cover going to feel like? Not just look like, but feel like. What are the materials of the product? How do I want the users to react when they pick it up? What type of paper am I using on the inside? Should I add ribbons so people can keep track of where they’re at in the process? All of these details add to the overall user experience, and they’re necessary ingredients in building a successful product.

That’s the wonderful thing about working with a team like Prouduct. They know what they’re doing. Even though the process has slowed, and beta testers have had to wait a bit, we’ve been keeping them up-to-date on where things are are at so that they’re not getting overly anxious. For you beta testers, if you haven’t received your products yet, you will very soon!

In the next installment of the Physical Product Experiment, the beta testers will already have started the process, making progress on their goal of writing the first draft of a book! As things come along, I’ll share more with you and look forward to revealing some numbers in terms of costs, shipping, warehousing, and all the things that come along with something like this.

Looking forward to it! Thanks so much for sticking around, and for paying attention. It means so much.

SPI 263: Laddering Up: How to Build a 7-Figure Business by Building a $7K per Month Business

Clay Collins of LeadPages returns to the Smart Passive Income Podcast for another incredible, value-packed (and potentially record-breaking) episode. This time, Clay is dropping serious knowledge on building a seven-figure business, step by step.
Durin…

SPI 262: Habit Forming Products with Nir Eyal, Author of Hooked

How many platforms do you check every day, almost without thinking about it? What products are an unconscious part of your daily routine? These kinds of habit forming products are what today’s episode is all about. Here to explore how customer…