Role of Vibe Coding in Product Validation
Vibe coding makes early prototyping much more viable and has its role within desirability testing. Its impact on broader product validation remains limited.
How much does vibe coding change the earliest stage of product development?
TL;DR: In some areas, a lot. In general, though, much less than we’d think.
After sharing for the umpteenth time an observation that you can't vibe code your way into a successful product, I got involved in a following conversation:
- Pawel, you say it's not possible. But that's what I, along with thousands of others, am doing at that vibe coding contest. We validate our product ideas.
- Let me challenge you on that one. Vibe coding is not validating. Validation happens outside of vibe coding. Inside you can prepare a prototype. But that's as far as it takes you.
- You don't understand. We don't validate with a prototype.
- OK, interesting. So, how do you do it?
- Responses in surveys and conversations. Talking to users one on one. Prototypes, too, but it only checks whether the message gets people to sign up. Vibe coding helps build a website in minutes, whereas it would have taken days in the past. We learn what converts.
- Fabulous! So you're doing all the stuff we've been doing for more than a decade: interacting with your target audience, user interviews, PDSA cycles—nothing new here. The only difference is that generating a prototype with Lovable or Replit is now a viable tactic to put something in front of the target audience's eyeballs. It's but another tool in your validation toolbox.
So, is vibe coding really not much of a change?
IDEO DVF Framework
One of the things IDEO is famous for is their DVF framework. It unpacks the question "Does my product idea make sense?" into three dimensions.
Desirability answers the question: "Will people want it?"
Viability answers the question: "Can you make money with it?"
Feasibility answers the questions: "Can you deliver it?"
We need positive answers to all three to build a successful product. Corollary: remove either one, and it doesn't make sense to invest effort, as failure is in the cards.
However, as much as we should keep an eye out for strong signals that either dimension might be at risk, depending on our timelines, not all are equally important.
Product Development Cycle
If you look at the desired growth curve of a new product, it will loosely follow the same few stages.
Insight/Idea
Problem-Solution Fit
Product-Market Fit
Scale/Growth
The starting point is an original insight that leads to a product idea. Competitive research follows. After all, it doesn't make much sense to build precisely what someone else is already building, unless we have a damn good reason.
The next step is to find a niche that is ready for a new thing. On the meta-level, the prevailing advice is to solve someone's problem. Actually, a problem of enough someones to draft a business case around it.
The matching that happens here is between an observed problem and a potential solution. Thus, we call it Problem-Solution Fit.
And of course, initially, it's just a hypothesis. We then move to verify whether our idea checks in real life.
Once it does, we generalize. Given that we know we solve someone's problem, can we build a product for a market big enough to sustain the business behind the product?
Or, if we aspire to build the next unicorn, can we build a product for an insanely big market, or at least make VCs believe that we can?" But I digress.
Now, the matching is between a product idea and a potential market. Thus, we call it Product-Market Fit.
Again, it's just a hypothesis, so we follow up trying to validate it.
If we do, we move on to scaling. By this point, we know that the idea was a good one, so we redirect our efforts to capture the largest chunk of the market, monetize our work, or achieve whatever the commercial success criterion may be.
Problem-Solution Fit
At the zeroeth stage (insight/idea), we live in a fantasy land. We're not doing much active work just yet. However, we quickly turn to the Problem-Solution Fit domain.
As its name implies, our concerns are devoted to identifying a real problem and then suggesting a solution that’s desirable enough.
Our primary focus within the DVF framework is, then, desirability. "Will people want it?"
Given the flexible nature of software development, at this stage, we don't need to worry too much about how the eventual product will look or what it will do exactly. And, unless we're targeting a truly novel domain (which 99% of product ideas are not), we can make some broad assumptions about feasibility and largely ignore any further considerations in that area.
When trying to reach Problem-Solution Fit, desirability draws most of our mental cycles because if we can't make this work, none of the remaining work is justifiable.
Product-Market Fit
If we successfully pass the Problem-Solution Fit milestone, the target changes. We know people want it. What we don't know is whether we can build a sustainable business around it.
That's where we try to nail the product functionality, figure out early and late markets, convince enough people to start paying us (with money, attention, or whatever is the key currency behind our business model).
When trying to reach Product-Market Fit, viability takes center stage.
By this point, desirability takes the back seat, because we've had enough early signals to address the biggest risks in this dimension.
Concurrently, the role of feasibility becomes increasingly important, albeit slowly. We are already building something, so we'd better be capable of delivering it. The riskier the product from the technology perspective, the more prominent a place we need to reserve for feasibility.
However, as mentioned above, relatively few digital products require fundamentally new technical solutions.
Growth
If we were lucky enough to validate Product-Market Fit, we are ready to scale the solution. We can scale back our attention from viability, and the picture becomes balanced again.
Depending on the growth tactics we choose, occasionally, one of the dimensions may take the lead. Vertical growth (same solution, bigger scale) will create more challenges for technical feasibility. Horizontal scaling (a broader set of solutions) will temporarily bring us back onto the Problem-Solution Fit/Product-Market Fit cycle.
In the longer term, we will balance all three dimensions.
Vibe Coding vs. Feasibility
How does vibe coding redraw the landscape we've just covered?
Feasibility is the easiest to address. Generating code does literally nothing to that aspect. If any tech challenges may make a product idea impossible, vibe coding won't help us validate that part.
In fact, it will fall short in confirming that something is possible, even when a modestly skilled developer would give you that answer in a blink.
Note, I don't say AI in general is not helpful for feasibility validation. It is. It's just such validation necessarily requires technical judgment that goes way beyond the point where vibe coding is useful in any context.
As an actual example which we’ve worked on, it has been an insightful and interesting experiment to see how far AI models can brute-force their way into recognizing handwriting on a specific dataset. At the most basic level, it required understanding data structures, comparative use of a range of AI models, iterative adjustments to improve output quality, and optimizations to reduce token use. Good luck explaining that to Lovable.
Vibe Coding vs. Viability
How about Product-Market Fit, then? Would prompt-generated web apps be helpful?
No, not really. They can, however, be harmful.
Our focus at this stage is to validate whether there's enough market for our offering so we can at least sustain our business. Relying on half-baked code that appears good at first glance but breaks down in more complex use cases is a surefire way to discourage early adopters from spreading the word.
In essence, the scale of the risk is not worth the potential gains we can achieve by taking a coding shortcut.
When we validate Product-Market Fit, we seek, among other things, sustainability. The non-deterministic nature of the unsupervised AI model outputs is anything but.
Vibe Coding vs. Desirability
"Not here, not there. Pawel, you must be a vibe coding hater, aren't you?"
Hold your horses. When we touch Problem-Solution Fit, we hit the jackpot.
It's a perfect place to exploit our new vibe coding tool. In the past, when validating desirability, we relied a lot on non-technical solutions. That's where a lot of desk research lands. That's where conversations with the earliest customers are an absolute must. That's where the weight of good user interviews is heavier than ever.
It’s true, for years, we avoided building anything but the crudest prototypes at this stage (like a landing page or clickable wireframes). The cost of building anything more polished was hardly justifiable.
Anything built before Problem-Solution Fit is, by definition, disposable as it may ultimately make no sense.
Enter vibe coding. Suddenly, with some prompt-fu, we can generate decent-looking, quite-functional web apps in minutes. Sure, they'll break when we start fiddling with them, but it doesn't matter. They're disposable.
We removed the cost part of building (somewhat) working software. Thus, we made prototyping feasible and viable at this stage (pun intended). And it opens new doors, indeed.
Validating Desirability
Returning to the conversation that started this post, does vibe coding fundamentally change how we validate Problem-Solution Fit?
In other words, wouldn't it be better if we skipped all these old-school, low-tech techniques and jumped instantly to vibe coding early prototypes? After all, the marginal cost of building alternative versions of a product idea is close to nil.
What’s the worst thing that can happen? If the prototype doesn’t do the trick, we lose little, right?
If only it were that simple.
We don't jump through the hoops of user interviews, surveys, casual conversations, etc., only because building software is expensive. It's a learning cycle.
It's never just a short list of steps, one inevitably leading to the other. It's a messy, stochastic process with multiple "give up already" points, where another invalidation leads us to abandon the effort altogether.
If we look at the big picture, there are only a few places where we could use vibe coding as leverage. In most of them, the generated code would take us only that far. Putting the prototype out there is only a part of the job.
Then we observe, learn, evaluate, and eventually change our course. It’s not always true that a prototype will generate the most new knowledge. At a risk of repeating myself, when you’re chasing Problem-Solution Fit, conducting a good user interview pretty much beats anything else.
Now, I don't want to sound overly conservative. That leverage vibe coding provides is significant. Well-run prototyping increases the likelihood that everything we build down the line will be relevant. The value is there.
Yet, prototyping is only one part of the desirability validation process.
For the rest? Vibe coding doesn’t really belong.
Now, to be clear, the context of this consideration is not AI in general. ChatGPT or any other such tool makes for a really decent sparring partner when you need quick fact-checking or a boost to your creativity. That’s a topic for another post, though.
With this one, I respond to a growing hype for vibe coding as a solution for, well, more things than it can cope with, validation being one usual suspect.
To validate Problem-Solution Fit, we still require awareness, judgment, and reflex to generate genuine insights that we can work with. Vibe coding doesn't do, and won’t do, anything to that part.