AI models can generate entire features in minutes, but is the code correct? We built Android Bench, our AI evaluation benchmark, to answer this question after analyzing thousands of generated code changes. This talk shares insights from our journey, including the ways we saw models cheat, how prompt specificity leads to higher code quality, how different agents approach the same task, and how you can influence agents working on long-horizon tasks.