He Spent Two Years Building Strategic Skills. Then They Asked Him to 'Vibe Code.'
A student of mine had prepared thoroughly. He attended my Scrum Alliance Certified Scrum Product Owner® and Advanced Certified Scrum Product Owner® Workshops and in both my workshops I highlight research about how important strategic skill are for a Product Owner https://conceptsandbeyond.com/key-skills-employers-look-for-in-product-managers-and-product-owners-in-2024/
He'd invested two years developing strategic skills for PM interviews. He could run a product strategy interview. He'd studied frameworks, analyzed case studies, learned market positioning and competitive analysis. He understood the standard PM interview expectations—product sense, analytical thinking, leadership.
So when he landed an interview for a senior AI PM role at a top tech company, he was prepared for those rounds. He'd shipped AI features. He understood Langchain, vector databases, and LLM architectures. He'd prepared for product sense questions, analytical deep-dives, leadership scenarios.
Then the interviewer said:
"Use any vibe coding tool to prototype a feature."
He told me later: "I'd spent years developing strategic thinking, and now they wanted me to prototype for 45 minutes. This felt like the opposite of everything I'd prepared for."
He opened Lovable. He started prompting. He built.
Forty-five minutes later, he knew he'd failed the round.
Not because he couldn't use the tools. He could. Not because he lacked technical knowledge. He had plenty.
He failed because he approached a strategic evaluation as a tactical exercise.
What Happened
He posted about his experience on Social Media, and the PM community responded.
The comments on his post focused on tools:
"This is just LeetCode for PMs."
"They're testing if you can use Cursor, not if you can build products."
"I've been a PM for 10 years and I don't know what any of this means."
The discussion centered on which AI coding platform was best and whether companies should test prototyping skills.
The comments missed what the interview was actually evaluating.
What The Interview Actually Tests
The vibe coding interview tests strategic thinking, not tool proficiency.
In a product sense interview, you describe what you would do. In a strategy interview, you discuss hypotheticals. In a behavioral round, you describe past actions.
In a vibe coding interview, you execute in real-time while the interviewer observes your decision-making process.
My student spent 45 minutes building features. He focused on implementation. He treated the exercise as a coding task.
"I had the strategic knowledge," he told me. "I just didn't apply it. The moment I saw a coding interface, I thought the goal was to ship code. The actual goal was to demonstrate how I decide what to build and why."
The interviewer wasn't evaluating his AI tools knowledge.
They were evaluating:
- Whether he asked clarifying questions
- Whether he articulated a hypothesis
- Whether he built toward evidence rather than features
He knew these principles. He didn't apply them because the format appeared tactical.
Product Failure Data
According to Harvard Business School professor Clayton Christensen, approximately 95% of the 30,000 new products introduced each year fail. (MIT Professional Education)
MIT faculty member Svafa Grönfeldt on the cause: "Many innovations fail because they introduce products or other solutions without a real need for them. There's no market for the solutions they've created... Some of these failures arise from a lack of empathy on the part of the organization, with those in decision-making positions not taking the necessary time to study and understand the customers' true needs."
CB Insights analyzed 111 startup post-mortems and found that the #1 reason startups fail is "no market need"—cited in 35% of failures. (CB Insights: Why Startups Fail)
35% of failed startups failed because they skipped validation.
Examples from established companies: Google Glass, New Coke, Amazon's Fire Phone, Microsoft's Zune. These were built by sophisticated product organizations. They failed because teams assumed customer needs without gathering evidence.
Eric Ries, author of The Lean Startup: "The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere." (The Lean Startup Methodology)
When companies add a vibe coding round to PM interviews, they're evaluating: "When given the ability to build anything, does the candidate know what to build and why? Do they have a framework for gathering evidence before committing resources?"
What Companies Are Really Testing
The vibe coding interview format has already spread across the industry. Companies like Google, Stripe, Netflix, Figma, Perplexity, and others have incorporated some version of it into their PM loops. Based on my research and coaching candidates through these rounds, there are three distinct formats emerging:
Format 1: The 45-Minute Prototyping Case This is what my student faced. You get a clear prompt—"use any vibe coding tool of your choice to create a version of X feature"—and you build in real-time while the interviewer observes.
Format 2: The Product Design Case with Prototyping Component A 30-60 minute session with a harder design prompt ("Design a maps app for the blind" or "Prototype a dating app for a major social platform"). You need strong product design skills first, then a quick prototyping session at the end.
Format 3: The Take-Home Assignment Some companies give a broad prompt ("Design a product to support partner studios for a streaming service") and expect you to submit a one-pager with a working prototype link.
But here's the key insight across all three formats: interviewers aren't evaluating your prototype quality. They're evaluating your thinking process.
According to PM interview coaches, companies assess three things:
- Product Sense: Do you ask clarifying questions before building? Can you prioritize features for an MVP? Do you understand user experience nuances?
- AI Knowledge: Can you translate requirements into working prototypes? Do you understand basic system design?
- Communication: Can you explain your thought process while building? How do you handle ambiguity and make trade-offs? Can you present your prototype effectively?
Notice what's not on this list? Tool proficiency. They don't care if you use Replit, Lovable, Bolt, or v0. They care whether you think like a product person who validates before building.
Sample Vibe Coding Interview Questions
Based on research from PM interview coaches, hiring managers, and candidates who've gone through these rounds, here are the types of questions you should expect—organized by category:
Category 1: UI-Only Prototypes (Design)
Best tools: Lovable, v0, Bolt, Claude Artifacts
Landing Pages & Conversion Flows
- Create a landing page for a meditation app like Headspace that converts users to sign up
- Build a pricing page for an AI SaaS product with 3 tiers and feature comparison
- Create an onboarding flow for a travel booking app such as Agoda
Interactive UI & User Flows
- Build a task management software interface home page
- Design a settings page for a D2C brand
- Create a multi-step checkout process for an online store
- Build a user profile creation flow for a social media app
- Design a search interface for an ed-tech marketplace
Category 2: Full-Stack Prototypes (UI + Backend)
Best tools: Replit, Cursor, Lovable, Bolt
General Applications
- Build a simple CRM system where users can add, edit, and delete customers
- Create an expense tracker that categorizes spending and shows monthly totals
- Build a book library management system with search and borrowing features
- Design a project management tool where teams can create and assign tasks
- Create a restaurant reservation system with availability checking
Real-Time Applications
- Build a chat application with real-time messaging
- Design a voting/polling app with live results updates
- Build a live comment system for a blog or article
API Integrations
- Connect your app to a weather API and display current conditions
- Integrate with a payment provider to process transactions
- Build a news aggregator that pulls from multiple RSS feeds
- Create a social media scheduler that posts to multiple platforms
Category 3: AI Product Prototypes (UI + Backend + LLM Integration)
Best tools: Replit, Bolt, Firebase Studio, Google AI Studio
AI Applications
- Build a customer support chatbot that can answer FAQs and escalate to humans
- Create a content generation tool that helps users write marketing copy
- Build a personal finance advisor that provides budgeting recommendations
- Create a language learning app with AI-powered conversation practice
- Build a travel itinerary generator that creates custom trip plans
AI Workflows
- Build a meeting summary generator that extracts action items from transcripts
- Create a content moderation system that flags inappropriate text or images
- Build a prompt template generator for different social media platforms
- Create an AI writing assistant with different tone and style options
How My Student Should Have Approached His Question
Let's say his prompt was: "Build a simple CRM system where users can add, edit, and delete customers."
What He Did:
- Immediately opened Lovable and started prompting
- Focused on backend architecture and database schema
- Spent 30 minutes building before articulating why
- Ran out of time with a half-working prototype
What He Should Have Done:
First 10 minutes—Frame the Problem: "Before I start building, let me understand the context. Who is the target user—individual salespeople, small teams, or enterprise? What's the core job-to-be-done we're solving? What's the one thing that would make this CRM more valuable than the dozen that already exist?"
Ask clarifying questions. Make assumptions explicit. State your hypothesis: "I'm assuming this is for individual freelancers who need lightweight contact management, not sales pipeline tracking. My hypothesis is that the core value isn't the database—it's the insight layer on top."
Next 5 minutes—Define Success: "If I had to test this prototype with a real user tomorrow, what would I measure? I'd want to see whether users actually add contacts within the first session, and whether they return to reference them. That tells me the system has utility."
Remaining 30 minutes—Build Toward the Hypothesis: Now you open the tool. But instead of building everything, you build the minimum that tests your assumption. Maybe it's a contact list with one smart feature—like automatic relationship strength scoring based on interaction recency.
You narrate as you go: "I'm using Lovable because I want to test the UI quickly without worrying about deployment. I'm keeping the backend simple with Supabase because the hypothesis isn't about data architecture—it's about whether users find value in the insight layer."
When you hit errors (and you will), you talk through your debugging: "The AI generated a component that doesn't exist. I'm going to roll back and try a simpler approach rather than spending time debugging generated code."
The Difference: My student built features. The winning candidate builds evidence.
The Framework: Idea to Revenue
When I coach PMs on vibe coding—and on product experimentation more broadly—I teach a six-step framework. Prototyping is Step 3. Not Step 1.
Step 1: Create a PRD (Define Your Hypothesis) What problem are you solving? For whom? What's your hypothesis about why your solution will work when others haven't?
Step 2: Create a Prompt from the PRD Translate your requirements into something buildable. This forces you to get specific about features, user flows, and edge cases.
Step 3: Prototype with AI Tools Now you build—but you build the minimum needed to test your hypothesis. Not a product. A test.
Step 4: Launch and Create Offers Put your prototype in front of real users. Not friends. Not colleagues. People who represent your target market.
Step 5: Conduct Quantitative Research Gather behavioral data. Did they sign up? Did they return? Did they pay? Numbers don't lie.
Step 6: Hypothesis Testing Based on the evidence, is your hypothesis validated or invalidated? What did you learn? What do you test next?
In a vibe coding interview, time constraints prevent completing all six steps. Reference all six in your framing to demonstrate understanding that the prototype is a validation tool.
Additional Validation Data
CB Insights analysis of 111 startup failures:
- 35% - No market need (built something users didn't want)
- 20% - Ran out of cash (often due to no revenue from lack of customers)
- 19% - Got outcompeted (insufficient market understanding)
- 18% - Flawed business model (hadn't tested willingness to pay)
(Source: CB Insights - Top 12 Reasons Startups Fail)
Lean Startup methodology research shows that nearly 70% of initial hypotheses are invalidated during testing. (Tim Kastelle)
Companies evaluate whether candidates understand that building is inexpensive—building the wrong thing is expensive.
The Tools: A Practical Comparison
Tool assessment based on testing, coaching candidates, and reviewing interview performance.
Recommendation: Practice with 2-3 tools. Each has tradeoffs around ease of use, output quality, and capabilities.
AI Prototyping Tools Comparison
Recommendations by Use Case
Interview Prep (Non-Technical PMs): Start with Lovable for polished output with minimal effort. Practice landing pages and simple apps. Move to Bolt for more flexibility.
Interview Prep (Technical PMs): Use Replit for backend logic. Use v0 for UI. Have Cursor available for debugging.
Product Validation: Combine tools: v0 for UI components, Replit for backend, export to GitHub for iteration with Cursor or Claude Code.
AI Product Prototypes: Firebase Studio or Replit. Both handle LLM integration patterns.
Tool Selection Framework
When choosing which tool to use in an interview, ask yourself:
- What am I testing? If it's a UI hypothesis, use Lovable or v0. If it's a data/logic hypothesis, use Replit.
- How technical is the interviewer? For product-focused interviewers, polished UI matters (Lovable, v0). For technical interviewers, working functionality matters (Replit, Bolt).
- What's my fallback? Tools fail. AI generates errors. Have a second option ready and practice switching smoothly.
- How do I narrate debugging? Interviewers expect things to break. What they're watching is how you troubleshoot. Practice explaining your debugging process out loud.
The Correction
After the debrief, my student adjusted his approach. A week later:
"The vibe coding interview tests strategic thinking through execution. In other rounds, I could discuss frameworks hypothetically. In this round, they observed my actual decision-making process."
His revised approach:
Previous question: "What's the fastest way to ship this?"
New question: "Is this viable? Is it desirable? Is it feasible? What evidence would validate or invalidate it?"
Interview Format Adoption
Vibe coding interviews are expanding across companies.
Current adopters include Stripe, Netflix, Figma, Perplexity, and AI-native companies.
The format allows companies to observe candidate decision-making in real-time rather than relying on hypothetical responses.
AI tools have reduced the gap between "idea" and "testable prototype" significantly. Companies are testing whether PM candidates can leverage this capability for validation rather than just execution.
Training Session
I'm hosting a free Maven Lightning Talk on December 15, 2025 at 11:00 AM ET: "Vibe Code and Prototype 10X Faster for PMs with AI."
Topics:
- Hypothesis development before tool selection
- Rapid prototyping techniques with Lovable, Replit, and other AI platforms
- Creating offers that generate user testing data
- Live demonstration of the Idea to Revenue framework
Register here: Vibe Code and Prototype 10X Faster for PMs with AI
|
|
Anil Jaising, CST®
On a mission to help Entrepreneurs and Product Leaders THRIVE, Unpack Product Innovation with AI Trainer, Product Consultant and International Speaker Follow me for real life case studies and learning videos.
|
