Overview

MPL was evaluating City of Crime: Gang Wars for the Indian market—a strategy title with rich narrative ambitions and layered gameplay. But before committing to publishing, they needed real player insights beyond Day 1 reactions.

• Would the game sustain interest over time?

• Would players discover and engage with its multiplayer mode?

Lysto was brought in to run a structured longitudinal study that could answer these questions decisively with AI-powered insights from real player feedback.

Challenge

MPL’s core question: Is this game engaging enough for the long run, or will players drop off after early novelty fades?

Key concerns included:

• Would the narrative and visuals hold up over repeated sessions?

• Would the strategic depth surface in time to hook players?

• Could players discover and engage with all key features, especially multiplayer?

A single-session playtest wouldn’t cut it.

"We needed to understand not just what players thought on Day 1, but what would keep them engaged—or drive them away—on Days 3, 7, and beyond. Longitudinal playtesting should give us these insights" explains our User Research Expert.

What is Longitudinal Playtesting?

Unlike traditional single-session evaluations, longitudinal studies follow the same participants over an extended period, allowing developers to observe how engagement patterns evolve. This methodology, borrowed from sociological research, is particularly valuable for products where long-term retention is critical to success.

Lysto's Playtest Strategy

We ran a 5-day longitudinal playtest using our AI-powered playtesting platform to simulate real-world gameplay:

• 100+ playtesters, matched to MPL’s audience segment (mobile strategy gamers)

• Daily 30-minute sessions, spread across multiple times of day

• Controlled experience with NDA, kill switch, and feedback lock-ins

• Feature discoverability analysis, using video review and player commentary to track natural discovery

• AI-assisted sentiment scoring, tagging friction points across time

The following were monitored:

• First gameplay session and feedback from the same

• Drop-offs and their reasons

• Subjective feedback on controls, gameplay, theme, and art

Lysto’s research team reviewed the gameplay recordings and the survey responses, combining firsthand behavioural observation with player-reported sentiment. This holistic approach helped uncover not just what players experienced but why it mattered.

Key Findings

1. Player journey analysis: Day 3 Cliff Observed

The retention curve told a clear story: An 81% drop-off across 5 days revealed what we now call the Day 3 Cliff — a point where the early novelty wore off and core mechanics no longer sustained engagement.

“They’d created an 'expertise cliff' where player skill outpaced game complexity", says our User Research Expert.

2. Strengths

• Visuals & Narrative: Players appreciated the gritty story, character art, and thematic consistency

• First 2 Days: Strong engagement, players were curious about the game world

• Initial Gameplay Loop: Well-received in early sessions, described as “immersive” and “cinematic”

3. Critical pain points

“Unlimited lives and bullets meant there was no challenge — I just breezed through.” - Playtester quote.

4. Focus group study discovery: Hidden multiplayer feature

Through the focus group discussion, Lysto uncovered a critical insight: the majority of players weren't aware of the game's multiplayer feature throughout the entire testing period.

Only one player discovered this feature, revealing a significant discoverability issue.

"We weren't informed by the game that we could collaborate with other players", noted one participant.

Another player on Day 3 commented: "Today, I felt that if it was a multiplayer game, it would have been better" - unaware that the feature actually existed.

Impact delivered

Based on Lysto's insights, the following were suggested as solutions:

1. Assess progression system tuning: Implement difficulty levels that progress with player skill and game advancement

2. Control enhancement: Add customizable combat options etc

3. Content diversification: Create diverse, non-linear missions with side-quests allowing more player freedom

4. Combat evolution: Increase power-ups, enemy types, and combat options for more strategic gameplay

Results

The playtest run in this fashion gave MPL confidence — not just in the game’s strengths, but in its shortcomings. The insights enabled a data-backed decision on whether to publish and a clear map of what to fix if they moved forward.

Conclusion

For publishers like MPL, early playtesting isn't just validation — it's risk mitigation. In an increasingly crowded market, understanding how player engagement evolves is key to making smart publishing bets.

The City of Crime longitudinal study shows that success isn’t about Day 1 excitement — it’s about Day 3 staying power.

If you’re a game publisher or developer with a title in the pre-production stage, want help to test concepts, or simply have any questions on how a playtest can improve your chances of a successful game, let’s talk.