Summary: AI and product management | Marily Nika (Meta, Google)

Staying informed on AI developments ▶

• Sources for AI news: Marily recommends following experts on Twitter and subscribing to any tech-focused newsletters, like the MIT Technology Review. “All tech newsletters would write AI-focused content going forward anyway.”
• AI is both overhyped and under-hyped: She thinks writers complaining writing is dying because of ChatGPT is overhyping and misunderstanding it. “AI is enhancing us, not stealing from us.” She thinks there is more to AI than just ChatGPT. Something that she found under-hyped in AI was news about how AI can now detect lies.

Using ChatGPT for work ▶

1. Writing mission statements that are readable by all stakeholders: Marily uses ChatGPT with a prompt of “Rewrite this mission statement for me,” and it produces something much better even on the first try, because it helps orient the words such that even kids and all stakeholders can understand and not just other PMs.
2. Creating user personas and framing the prompt: It provides her with user segments that you might not even think of, while providing motivations, pain points, etc. To frame a persona prompt, she mentions an example: if you were responsible for creating a fitness band that doesn’t have a screen, you can ask something like “Who would be interested in a fitness band that doesn’t have a screen?” and it provides a list, like people who don’t want to charge it often, among others.
3. Providing AI-enhanced ideas: She doesn’t make it do her job for her, but when she already has an issue in her hand, she asks it to improve it with a lot of ideas.

Why all PMs will be AI PMs ▶

• All PMs will be AI PMs because she sees all products needing to have personalized experiences and recommender systems, and even to make technological advancements, you need to have an AI-centric view.
• PMs should get comfortable with having research scientists on their teams. A lot of PMs are very used to saying, “Okay, I’m going to do XYZ on launch” and don’t know how to approach researchers. Whereas when you’re working with research, it’s more like: we’re going to try this, and then in a year if it doesn’t work out, we’re going to shut the whole thing down.
• Find the intersection of something that’s desirable by users, something that is going to be a viable business, and something that is going to be feasible from a research-scientist and technical perspective.

How to get started using AI ▶

• Think twice before you use AI for your MVP. If you need to just get buy-in for an idea that may use AI in the future, just create a Figma prototype and a demo of what the experience is going to be like.
• Anything where you can get data behind the behavior with users can be improved with AI. For every product you ship, you can always ask yourself if it can be made smarter, e.g. enhancing it with fraud detection; if it’s in health-care you can make it faster or more accurate, etc. Leverage the data that is just sitting around.
• Don’t fall for the “shiny object trap.” Make sure there is a problem and pain point that needs to be solved in a smart way. Once you figure that out, only then figure out how to implement it (by hiring a data science intern or some other way). Only use AI when you already have some data, or data from an adjacent product.

How much data you need for AI to work

• If you’re trying to do classification (like a photo being of a cat or a dog), even 20 labeled photos might work, but if you’re making complicated NLP products or voice recognizers, etc., you might need thousands of data.

When to build your own model vs. use a ready-made model ▶

• A model is like a 3-year-old kid’s brain: You will end up training your kid’s brain by repeating the same information again. Saying, “Here’s what the rhino looks like, here’s what an elephant looks like,” etc., repeatedly. And once you’ve done this enough times, then your kid will see the animal and they’ll be able to recognize it. This is how models work as well.
• Most impactful AI demo she saw: Google demonstrated the power of AI by translating a conversation between two people speaking different languages in real time via Google Glass.
• Build tools only if existing ones don’t meet your needs: Companies should consider developing their own AI tools when off-the-shelf solutions don’t meet their specific needs or when they have a competitive advantage in doing so. This would probably be more relevant for larger companies.
• Most startups also don’t have enough data to build their own models and should rely on using off-the-shelf models more.
• An AI PM role is to decide when the recognition level is good enough to launch the product. Big tech companies should use diverse data to train their model to get better-quality than competitors’.

How to become a strong AI PM ▶

• AI will let PMs focus on some things more than others: AI augments PM capabilities, making them more efficient and effective. PMs should focus less on areas that can be managed/handled with machine learning/AI and more on building their skill sets to understand the underlying mechanisms; e.g., for Marily, having GPT write for her across projects lets her focus her time on more strategic things.
• AI PMs should learn how to code. Even though ChatGPT etc. can help code, learning it gives you a different mindset and confidence, knowing how things work; e.g. in learning to play the piano, Marily hated that she had to learn classical music first, but she understood from her teacher that she needed to learn fundamentals that would help her later.
• Where to learn: Marily recommends her AI PM Bootcamp or AI PM 101 or Advanced course but also courses on Coursera like Intro to AI, CareerFoundry, General Assembly, and Coding Dojo. If your company already has AI projects, talk to people on those teams. PMs should also read papers on Arxiv. Lenny recommended Tyler Cowen’s blog, called Marginal Revolution, to read research paper insights.
• Strong AI PM qualities: To become a strong AI PM, one should have a good understanding of AI, be able to work with data scientists, and possess excellent communication skills. Marily has a course on this for PMs and aspiring PMs.
• Ask yourself “Is there a problem that can be solved with a smart solution?” but make sure you start from the problem itself.

4 main challenges for AI PMs ▶

1. Dealing with uncertainty: With your given hypothesis, when you actually train the model, the results you may be getting may not be answering the questions or the hypothesis that you actually had in mind.
2. Leadership support: You might have to pivot, so leading that change can be difficult, and you need to be prepared.
3. Getting good data is hard, so be willing to do everything: You may need to figure out creative ways to get the data, e.g. maybe going to the street and asking people to contribute data for you.
4. Career trajectory for AI PMs is not defined by launches: Usually PMs get ahead the more they launch. So AI PMs need to clarify with hiring managers early on what progress means and how you will be assessed in research work.

Getting leadership buy-in for investing in AI ▶

• Use examples of successful adjacent products. Marily reminds the leadership about how a previous project in the same space (adjacent product) sounded crazy at the time but has worked out well.
• Provide contingency and rollback plans. In case the project doesn’t work out, have the rollback plan and mention the maximum downside we could have in doing the project.
• PMs need to actually figure out ways to also monetize the ideas of research scientists, e.g. there was a willingness-to-pay survey for ChatGPT to try to monetize it and PMs have to come bridge that gap.

Marily’s AI course and AutoML ▶

• Marily’s three-week AI course covers the product development lifecycle, idea generation, production, and career paths in AI. Students learn coding fundamentals and no-code tools while developing their own AI products. Some students have even raised funding for their creations. People have built a tool to diagnose a patient based on an x-ray in three weeks and managed to raise funding after the course as well.
• AutoML as a no-code AI tool: AutoML is a tool offered by Google Cloud that allows for the training of custom machine learning models with minimal effort. A renewable-energy company was able to use AutoML to reduce their turbine maintenance procedure from three weeks to a few hours using AutoML.
• Marily treated her course like a product, e.g. the first version was directed toward software engineers, but she later pivoted to a less technical audience based on market reception. She continually updates and modifies her course based on feedback and new developments in the AI field.

The importance of creating your own course

• Course creation: Marily built her course using her experience in AI and product management, as well as feedback from students and industry professionals.
• Why create a course: She believes that creating a course is a great way to share knowledge, improve one’s own understanding, and help others learn about a topic.
• Her favorite interview question is “explaining complex concepts”: In interviews, Marily likes to ask candidates how they would explain a database to a 3-year-old.

• Marily recommends ChatGPT and the Lensa app, which allows users to transform their photos into fantastical images.

This is a human edited summary of the podcast episode with Marily, by Gaurav Chandrashekar (@cggaurav, productscale.xyz). To listen to the full episode, go here.