Skip to main content

Create themes

Problem​

We don't want to generate User Preferences anymore but rather Themes.

Control points​

  • Score of pertinence given by PM

Answer​

Themes that are generated :

  • the themes are almost perfect on the first prompt (5/5 by Justine Storelli - V2 on the Michelin analysis)
  • the themes are too large and imprecise when merging multiple prompt together (V1 on the Michelin analysis) Descriptions :
  • they induce the PM in error => does not precise if the associated reviews are good or bad

Solution​

A new prompt​

Specify themes.

You are an experienced Product Manager. You are tasked with finding
five main themes from the following reviews, that should allow to
compare all apps against each other.
DO NOT make generic themes (such as "Usability", "Design" or "Features");
instead focus on business-specific themes (such as specific use cases).

A new output schema​

Ask for a description to better assign reviews later.

"type": "array",
"description": "List of themes relevant to the apps.",
"items": {
	"type": "object",
	"properties": {
		"label": {
			"type": "string",
			"description": "A short label of the theme",
		},
		"key": {
			"type": "string",
			"description": "The label in snake_case format.",
		},
		"description": {
			"type": "string",
			"description": (
				"Explain in details what this theme represents."
			),
		},
		"relative_importance": {
			"type": "number",
			"description": (
				"The importance of this theme relative to other themes"
			),
		},
	},
},

Limitations​

  • the descriptions can let think that all reviews associated are good or bad
    • => we should hide the description to the user?
    • => we can also make a GPT-analysis for each theme and extract good and bad points, with associated comments
  • the quality of the association between themes and reviews has not been evaluated
  • merging themes seems to make worse themes :
    • => update the merging prompt to allow more resulting themes and keep the same granularity, instead of trying to merge all themes into 5 themes only?

Current prompt for merging:

"You already found themes for multiple sub-set "
"of the full list of reviews. Now you have to to merge "
"this list of theme lists, to obtain a single "
"list of 5 to 6 themes. Do not pick only some "
"themes, try instead to merge similar ones together, "
"so as to respect the limit."