Project description

Toad for Oracle a desktop application for managing Oracle databases.  For decades, this product was offered as freeware as well as a commercial product.  When the Product Managers were exploring not offering the freeware version, they had numerous questions about these users and how they compared to the commercial, Base edition users.  I led a survey to provide answers to these questions.

Research objective

Learn about the Freeware user community:

• Who are our Freeware users?
• How do they differ from our Base edition users?
• What might we do different in a new, commercial offering to replace Freeware?

Research method

I used a survey with a combination of questions to learn about users. The survey used UMUX-lite usability ratings, rating scale questions, multiple-choice questions, and short answer questions.

I asked demographic information, including years in their current role, age, and company size.

I asked about their tool usage, including how invested they are in their current tools, how long they have used our product, whether it is used for personal or business purposes, and what other database tools they regularly use.

I asked about their feelings toward our product and asked them to rate its usability using the UMUX-lite metric.

I asked them to describe one thing they particularly like our product, as well as one thing they would change if they had a magic wand. 

High level findings and recommendations

Analyzing the survey responses, Freeware users tended to be younger and belong to smaller companies, while Base edition users were more experienced with the product.

Contrary to our internal beliefs, Freeware users were not opposed to switching tools. They also mentioned the limitations their current version as improvement suggestions.

There was no consensus on the most “problematic” Freeware limitation, though, as numerous ideas were provided. More investigation was recommended to identify a priority.

The Freeware and Base edition user responses were very similar:

• Both rated it both highly useful (4.6 out of 5) and usable (4.3 out of 5)
• Both reported using an average of 1.5 other database tools alongside our software
• Both raised concerns about software stability, and performance issues.

Price was also a big concern for both types of users. Another recommendation was to evaluate the pricing model with users, especially those transitioning from freeware.

Participants

Recruiting

To recruit Freeware users, I contacted the unique email addresses associated with a Freeware download from the previous 8 months. 

To recruit Base edition users, I used contacts provided by our internal Sales team as well as the unique email addresses associated with online purchases of the product from the previous 8 months.

Response rate

I received 30 responses from Freeware users after sending 6889 emails (0.45% response rate).  I had 52 responses from the Base edition users.

Incentive

I did not offer an incentive to our customers.

Analysis process

For the multiple-choice questions, I tallied the responses to understand how many participants chose each option. I considered the most frequently selected and least frequently selected options. I created visualizations to analyze trends in the data. I also compared Freeware and Base user responses to look for similarities and differences.

For the rating-scale questions, I calculated averages as well as looked at the distribution across the rating options.

I also reviewed all of the responses to short answer questions to identify common responses and themes.

Share out

Numerous stakeholders were interested in the results of the study. The change in our product strategy, moving away from Freeware, was significant. Product management, engineers, designers, marketing, and the sales group were curious about our Freeware users and how we might create a commercial product for them.

To share the findings, I prepared a formal report – Freeware and Base edition Users (send a request to access it). This was presented in a series of meetings to various stakeholders.

Impact

This survey provided the Product Managers insights into the Freeware user community as they debated ending the freeware product. They used this information to finalize their decision and create a promotional campaign to switch to another edition. With many improvement suggestions, the Product Managers also had additional data to inform their product roadmap.