Guests: Aishwarya Naresh Reganti - Early AI researcher at Alexa and Microsoft, published over 35 research papers. Kiriti Badam - Works on Kodex at OpenAI, with a decade of experience in AI and ML infrastructure at Google and Kumo. Together, they have led over 50 AI product deployments across major tech companies and teach a top-rated AI course on Maven. Key Takeaways: Non-Determinism in AI: AI products are non-deterministic, meaning both user inputs and AI outputs can vary, requiring a new approach to product development. Agency vs. Control: Start with high control and low agency in AI systems, gradually increasing agency as confidence in the system's reliability grows. Iterative Development: Use a continuous calibration and development framework to iteratively improve AI products, focusing on behavior calibration without losing user trust. Leadership and Culture: Successful AI product development requires leaders to be hands-on and open to learning, fostering a culture of empowerment rather than fear of replacement. Evals and Monitoring: Both evaluation datasets and production monitoring are crucial for understanding and improving AI product performance. Topics Covered: Non-determinism in AI, agency vs. control, iterative development, leadership in AI, AI product lifecycle, evals and monitoring, AI product success factors.
