[{"id":"fe6b57f3-6ddf-4c53-aac3-16112e956614","user_id":"f8b58e56-461e-4f06-b673-b1d219056532","question":"what do you think is the future of publishing in an age of AI?","text":"The future of publishing in the age of AI is going to be quite different from what we've known. Based on my experiences and observations, here are some thoughts on how it might evolve:\n\n[Using AI-powered systems like Ask Luke has made me reconsider the nature of content creation and publishing. Instead of producing discrete articles, we're moving towards creating a malleable, unified content corpus that can be accessed and utilized as needed.](3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a)\n\n[One significant change I've noticed is the ability to generate content more fluidly. For instance, with the Ask Luke system, when a question comes in that I don't have a pre-existing answer for, I can simply hit record and talk. This gets incorporated into the content corpus without the need for formal writing or editing.](2a247159-cf3d-4e22-8297-0425f8cf87d7) [It's a very different approach to content strategy - less structured and more responsive to real-time needs.](2a247159-cf3d-4e22-8297-0425f8cf87d7)\n\n[This shift has led me to generate the equivalent of about 500 articles in a year and a half - something that would be nearly impossible in traditional publishing.](2a247159-cf3d-4e22-8297-0425f8cf87d7) It's changing the pace and volume of content creation dramatically.\n\nHowever, this new paradigm raises questions about content quality and curation. [There's a balance to strike between the ease of fluid content generation and maintaining high-quality, polished output. Currently, I lean towards maintaining quality, but it's a question worth continually revisiting.](3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a)\n\nAnother interesting aspect is how AI is changing content consumption and format. [While we're moving towards more dynamic, AI-generated content, there's still demand for traditional formats. For instance, people still request PDFs of AI-generated answers, or ask for content to be turned into podcasts.](9e728f85-426f-4055-bf2a-7434ec08a366) This suggests that while the creation process is changing, some consumption habits remain rooted in familiar formats.\n\nLastly, it's crucial to consider the accuracy and reliability of AI-generated content, especially in fields like finance where incorrect information could have serious consequences. [This is why I built the Ask Luke system on my own content corpus - it allows me to verify the accuracy of the output.](9e728f85-426f-4055-bf2a-7434ec08a366) As AI becomes more prevalent in publishing, developing robust methods for ensuring accuracy and combating misinformation will be essential.\n\nIn essence, the future of publishing with AI looks to be more dynamic, responsive, and voluminous, but it will also require new approaches to quality control, format flexibility, and accuracy verification.","message":{"id": "fe6b57f3-6ddf-4c53-aac3-16112e956614", "type": 0, "title": "what do you think is the future of publishing in an age of AI?", "answer": "The future of publishing in the age of AI is going to be quite different from what we've known. Based on my experiences and observations, here are some thoughts on how it might evolve:\n\n[Using AI-powered systems like Ask Luke has made me reconsider the nature of content creation and publishing. Instead of producing discrete articles, we're moving towards creating a malleable, unified content corpus that can be accessed and utilized as needed.](3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a)\n\n[One significant change I've noticed is the ability to generate content more fluidly. For instance, with the Ask Luke system, when a question comes in that I don't have a pre-existing answer for, I can simply hit record and talk. This gets incorporated into the content corpus without the need for formal writing or editing.](2a247159-cf3d-4e22-8297-0425f8cf87d7) [It's a very different approach to content strategy - less structured and more responsive to real-time needs.](2a247159-cf3d-4e22-8297-0425f8cf87d7)\n\n[This shift has led me to generate the equivalent of about 500 articles in a year and a half - something that would be nearly impossible in traditional publishing.](2a247159-cf3d-4e22-8297-0425f8cf87d7) It's changing the pace and volume of content creation dramatically.\n\nHowever, this new paradigm raises questions about content quality and curation. [There's a balance to strike between the ease of fluid content generation and maintaining high-quality, polished output. Currently, I lean towards maintaining quality, but it's a question worth continually revisiting.](3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a)\n\nAnother interesting aspect is how AI is changing content consumption and format. [While we're moving towards more dynamic, AI-generated content, there's still demand for traditional formats. For instance, people still request PDFs of AI-generated answers, or ask for content to be turned into podcasts.](9e728f85-426f-4055-bf2a-7434ec08a366) This suggests that while the creation process is changing, some consumption habits remain rooted in familiar formats.\n\nLastly, it's crucial to consider the accuracy and reliability of AI-generated content, especially in fields like finance where incorrect information could have serious consequences. [This is why I built the Ask Luke system on my own content corpus - it allows me to verify the accuracy of the output.](9e728f85-426f-4055-bf2a-7434ec08a366) As AI becomes more prevalent in publishing, developing robust methods for ensuring accuracy and combating misinformation will be essential.\n\nIn essence, the future of publishing with AI looks to be more dynamic, responsive, and voluminous, but it will also require new approaches to quality control, format flexibility, and accuracy verification.", "resources": [{"id": "3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a", "url": "https://www.lukew.com/ff/entry.asp?2082", "type": 1, "title": "Publishing in the Generative AI Age", "summary": "The author, Luke Wroblewski, explains his new approach to publishing content in the age of generative AI. He uses a feature on his site called \"Ask Luke\" to answer questions about digital product design using his past works. When a question can't be answered, he uses a feature called \"saved questions\" to add information for future reference. Since launching the feature in April 2023, he has added nearly 500 saved questions to his content corpus. This content can be weighted to influence future answers. He debates whether to publish these saved questions as blog posts or keep them behind the scenes, and how much polish they need. He also discusses the concept of a malleable content corpus versus individual articles, suggesting a shift in the traditional publishing paradigm.", "entities": [], "metadata": {"tags": ["The Death of Lorem Ipsum", "Ask LukeW: Integrated Video Experiences", "Ask LukeW: Custom Re-ranker", "ai", "publishing", "content", "ai design"], "title": "LukeW | Publishing in the Generative AI Age", "author": "Luke Wroblewski", "longDate": "January 7, 2025", "shortDate": "Jan 7, 2025"}, "coverImage": "https://static.lukew.com/lukew_newpublish1.png", "renderHtml": "

Hey Luke, why aren't you publishing new content? I am... but it's different in the age of generative AI. You don't see most of what I'm publishing these days and here's why.

\n\n

The Ask Luke feature on this site uses the writings, videos, audio, and presentations I've published over the past 28 years to answer people's questions about digital product design. But since there's an endless amount of questions people could ask on this topic, I might not always have an answer. When this happens, the Ask Luke system basically tells people: \"sorry I haven't written about this but here's some things I have written about.\" That's far from an ideal experience.

\n\n

$\"Ask$

\n\n

But just because I haven't taken the time to write an article or create a talk about a topic doesn't mean I don't have experiences or insights on it. Enter \"saved questions\". For any question Ask Luke wasn't able to answer, I can add information to answer it in the future in the form of a saved question. This admin feature allows the corpus of information Ask Luke uses to expand but it's invisible to people. Think of it as behind-the-scenes publishing.

\n\n

$\"Ask$

\n\n

Since launching the Ask Luke feature in April 2023, I've added close to 500 saved questions to my content corpus. That's a lot of publishing that doesn't show up as blog posts or articles but can be used to generate answers when needed.

\n\n

Each of these new bits of content can also be weighted more or less. With more weight, answers to similar questions will lean more on that specific answer.

\n\n

$\"Ask$

\n\n

Without the extra weighting, saved questions are just another piece of content that can be used (or not) to answer similar questions. You can see the difference weighting makes by comparing these two replies to the same question. The first is weighted more heavily toward the saved question I added.

\n\n

$\"Ask$

\n\n

Using this process triggered a bunch of thoughts. Should I publish these saved questions as new articles on my blog or keep them behind the scenes? What level of polish do these types of content additions need? On one hand, I can simply talk fluidly, record it, and let the AI figure what to use. Even if it's messy, the machines will use what they deem relevant, so why bother? On the other hand, I can write, edit, and polish the answers so the overall content corpus is quality is consistently high. Currently I lean more toward the later. But should I?

\n\n

Zooming up a level, any content someone publishes is out of date the moment it goes live. But generated content, like Ask Luke answers, are only produced when a specific person has a specific question. So the overall content corpus is more like a fully malleable singular entity vs. a bunch of discrete articles or files. Different parts of this corpus can be used when needed, or not at all. That's a different way of thinking about publishing (overall corpus vs. individual artifacts) with more implications than I touched on here.

", "textOnlyHtml": "Hey Luke, why aren't you publishing new content? I am... but it's different in\nthe age of generative AI. You don't see most of what I'm publishing these days\nand here's why.\n\nThe Ask Luke feature on this site uses the writings, videos, audio, and\npresentations I've published over the past 28 years to answer people's questions\nabout digital product design. But since there's an endless amount of questions\npeople could ask on this topic, I might not always have an answer. When this\nhappens, the Ask Luke system basically tells people: \"sorry I haven't written\nabout this but here's some things I have written about.\" That's far from an\nideal experience.\n\n\n\nBut just because I haven't taken the time to write an article or create a talk\nabout a topic doesn't mean I don't have experiences or insights on it. Enter\n\"saved questions\". For any question Ask Luke wasn't able to answer, I can add\ninformation to answer it in the future in the form of a saved question. This\nadmin feature allows the corpus of information Ask Luke uses to expand but it's\ninvisible to people. Think of it as behind-the-scenes publishing.\n\n\n\nSince launching the Ask Luke feature in April 2023, I've added close to 500\nsaved questions to my content corpus. That's a lot of publishing that doesn't\nshow up as blog posts or articles but can be used to generate answers when\nneeded.\n\nEach of these new bits of content can also be weighted more or less. With more\nweight, answers to similar questions will lean more on that specific answer.\n\n\n\nWithout the extra weighting, saved questions are just another piece of content\nthat can be used (or not) to answer similar questions. You can see the\ndifference weighting makes by comparing these two replies to the same question.\nThe first is weighted more heavily toward the saved question I added.\n\n\n\nUsing this process triggered a bunch of thoughts. Should I publish these saved\nquestions as new articles on my blog or keep them behind the scenes? What level\nof polish do these types of content additions need? On one hand, I can simply\ntalk fluidly, record it, and let the AI figure what to use. Even if it's messy,\nthe machines will use what they deem relevant, so why bother? On the other hand,\nI can write, edit, and polish the answers so the overall content corpus is\nquality is consistently high. Currently I lean more toward the later. But should\nI?\n\nZooming up a level, any content someone publishes is out of date the moment it\ngoes live. But generated content, like Ask Luke answers, are only produced when\na specific person has a specific question. So the overall content corpus is more\nlike a fully malleable singular entity vs. a bunch of discrete articles or\nfiles. Different parts of this corpus can be used when needed, or not at all.\nThat's a different way of thinking about publishing (overall corpus vs.\nindividual artifacts) with more implications than I touched on here."}, {"id": "2a247159-cf3d-4e22-8297-0425f8cf87d7", "url": "https://lukew_img.s3.amazonaws.com/video/smashingconfOct2024_howaiatemywebsite720.mov", "date": 0, "type": 2, "title": "Discussing the Integration and Future Potential of AI in Products and Websites", "topics": ["AI integration", "Product development", "Web form design", "Mobile technology", "AI discussion", "AI models", "Conversational interfaces", "Content cleanup", "Data pipelines", "Website design", "Language models", "Vision models", "Inline citations", "Inline video files", "AI content strategy", "AI application models", "AI in healthcare", "AI testing for accuracy", "Artificial General Intelligence (AGI)"], "endTime": 2979.161, "nameMap": {"undefined": "Luke W", "SPEAKER_00": "Luke Wroblewski", "SPEAKER_01": "Speaker A", "SPEAKER_02": "Luc"}, "summary": "The conversation is about the integration and relevance of Artificial Intelligence (AI) in products and websites. It discusses the use of AI models to process different types of content and answer user queries in a personalized and efficient manner. The conversation also touches on the continuous adjustment of AI models, the transformation of traditional website interfaces into interactive ones, and the future potential of AI in technology. The speakers, Luke Wroblewski and Luc, share their experiences and views on the development and testing of AI systems.", "filename": "149e49b5-1b4c-44d0-8af9-9be24815d005-smashingconfOct2024_howaiatemywebsite720", "startTime": 2873.248, "coverImage": "/fallback-audio-cover.png", "transcript": [{"end": 1.591, "text": " the last session of the day.", "start": 0.009, "speaker": "SPEAKER_00"}, {"end": 2.913, "text": "And we have Luke with us.", "start": 2.072, "speaker": "SPEAKER_00"}, {"end": 4.976, "text": "Look, lurking in the back.", "start": 2.953, "speaker": "SPEAKER_00"}, {"end": 6.719, "text": "Hello, Luke.", "start": 6.138, "speaker": "SPEAKER_00"}, {"end": 8.422, "text": "Hello.", "start": 7.981, "speaker": "SPEAKER_00"}, {"end": 10.384, "text": "So we both are green.", "start": 9.443, "speaker": "SPEAKER_00"}, {"end": 12.696, "text": " Yes, you finally caught up.", "start": 11.536, "speaker": "SPEAKER_02"}, {"end": 14.677, "text": "We finally like this is like you match the slides.", "start": 12.716, "speaker": "SPEAKER_00"}, {"end": 15.657, "text": "So did we?", "start": 14.917, "speaker": "SPEAKER_00"}, {"end": 17.577, "text": "Yeah, the color partially.", "start": 16.017, "speaker": "SPEAKER_00"}, {"end": 18.537, "text": "Yeah, there is a bit of that.", "start": 17.597, "speaker": "SPEAKER_00"}, {"end": 20.498, "text": "So AI, huh?", "start": 19.177, "speaker": "SPEAKER_00"}, {"end": 21.718, "text": "Still a thing.", "start": 21.258, "speaker": "SPEAKER_00"}, {"end": 22.938, "text": "I was I think it's a thing.", "start": 21.998, "speaker": "SPEAKER_00"}, {"end": 24.478, "text": "Somebody mentioned it earlier today.", "start": 23.058, "speaker": "SPEAKER_00"}, {"end": 24.918, "text": "Yeah.", "start": 24.618, "speaker": "SPEAKER_00"}, {"end": 25.178, "text": "Yeah.", "start": 24.978, "speaker": "SPEAKER_00"}, {"end": 28.479, "text": "So we've been having conversation about, uh, well, many things yesterday.", "start": 25.218, "speaker": "SPEAKER_00"}, {"end": 35.06, "text": "And then of course the AI came along and, oh my, look is all in, like all in 100% in AI is this.", "start": 28.499, "speaker": "SPEAKER_00"}, {"end": 36.56, "text": "So what do you do?", "start": 35.82, "speaker": "SPEAKER_00"}, {"end": 40.841, "text": "Anybody actually working on a product that now embeds or integrates AI in some way?", "start": 36.62, "speaker": "SPEAKER_00"}, {"end": 42.831, "text": " A few people?", "start": 42.31, "speaker": "SPEAKER_00"}, {"end": 43.631, "text": "Okay.", "start": 42.851, "speaker": "SPEAKER_00"}, {"end": 45.073, "text": "Who has given up on AI?", "start": 43.892, "speaker": "SPEAKER_00"}, {"end": 46.194, "text": "Like AI is like so 2022.", "start": 45.093, "speaker": "SPEAKER_00"}, {"end": 46.494, "text": "Okay.", "start": 46.294, "speaker": "SPEAKER_00"}, {"end": 53.061, "text": "A few people, a few people, maybe, maybe look will change your mind today as well.", "start": 46.554, "speaker": "SPEAKER_00"}, {"end": 58.266, "text": "Apparently, AI has eaten, well, looks website.", "start": 53.401, "speaker": "SPEAKER_00"}, {"end": 59.366, "text": " Don't give it away.", "start": 58.766, "speaker": "SPEAKER_00"}, {"end": 60.207, "text": "I should.", "start": 59.907, "speaker": "SPEAKER_00"}, {"end": 60.627, "text": "Okay.", "start": 60.367, "speaker": "SPEAKER_00"}, {"end": 61.107, "text": "I will not.", "start": 60.707, "speaker": "SPEAKER_00"}, {"end": 62.228, "text": "So, but let's see.", "start": 61.147, "speaker": "SPEAKER_00"}, {"end": 69.172, "text": "I mean, again, looking at what we, not, not what we can do with the AI, but maybe what we should do with the AI and how it all works under the hood.", "start": 62.268, "speaker": "SPEAKER_00"}, {"end": 73.754, "text": "And also just for you to know, of course, let me just drop, drop in back in here.", "start": 69.652, "speaker": "SPEAKER_00"}, {"end": 85.021, "text": "So Lucas Karanti, Managing Director, Sato Whittle Hill Ventures, but also earlier was a product director at Google and spent a lot of time of his life with little things like web form design.", "start": 74.475, "speaker": "SPEAKER_00"}, {"end": 85.681, "text": "Remember that?", "start": 85.061, "speaker": "SPEAKER_00"}, {"end": 86.221, "text": "Yeah, I do.", "start": 85.721, "speaker": "SPEAKER_00"}, {"end": 86.221, "text": "2010.", "start": 86.241, "speaker": "SPEAKER_00"}, {"end": 86.742, "text": "Irrelevant now.", "start": 86.281, "speaker": "SPEAKER_00"}, {"end": 89.921, "text": " And mobile?", "start": 89.241, "speaker": "SPEAKER_02"}, {"end": 91.682, "text": "Yeah, mobile, also irrelevant.", "start": 90.221, "speaker": "SPEAKER_02"}, {"end": 96.063, "text": "Well, then, I don't know what to expect now.", "start": 93.082, "speaker": "SPEAKER_00"}, {"end": 97.704, "text": "Well, AI, I guess.", "start": 96.223, "speaker": "SPEAKER_00"}, {"end": 100.024, "text": "So, please, round of applause to Lukro Blewski.", "start": 98.104, "speaker": "SPEAKER_00"}, {"end": 106.987, "text": "Okay, thank you for having me.", "start": 100.044, "speaker": "SPEAKER_02"}, {"end": 111.168, "text": "And, uh, there we go.", "start": 108.667, "speaker": "SPEAKER_02"}, {"end": 114.069, "text": "And all of my little AI guys.", "start": 112.568, "speaker": "SPEAKER_02"}, {"end": 124.572, "text": " Now, the staff here is so on top of it because we blew up these balloons and the moment you guys walked out the door, they took them all away.", "start": 116.747, "speaker": "SPEAKER_02"}, {"end": 132.138, "text": "And I had an elaborate scheme for all of us stomping on the balloon at the same time and making a big noise.", "start": 125.613, "speaker": "SPEAKER_02"}, {"end": 134.479, "text": "And now that has completely fallen apart.", "start": 132.998, "speaker": "SPEAKER_02"}, {"end": 134.8, "text": "Thank you.", "start": 134.499, "speaker": "SPEAKER_02"}, {"end": 137.122, "text": " But we're going to try this.", "start": 135.6, "speaker": "SPEAKER_02"}, {"end": 138.203, "text": "Can you stand up for a second?", "start": 137.202, "speaker": "SPEAKER_02"}, {"end": 139.665, "text": "You've been sitting really long.", "start": 138.223, "speaker": "SPEAKER_02"}, {"end": 140.766, "text": "Your butt's probably hurt.", "start": 139.705, "speaker": "SPEAKER_02"}, {"end": 142.308, "text": "Your brains are fried.", "start": 141.246, "speaker": "SPEAKER_02"}, {"end": 148.274, "text": "Okay, so instead of stomping on a balloon, just try to make a big stomp noise.", "start": 142.328, "speaker": "SPEAKER_02"}, {"end": 149.976, "text": "Okay, we have to time it, all right?", "start": 148.575, "speaker": "SPEAKER_02"}, {"end": 151.017, "text": "Ready?", "start": 150.777, "speaker": "SPEAKER_02"}, {"end": 154.701, "text": "So I'm here to talk about how AI ate my website, right?", "start": 151.618, "speaker": "SPEAKER_02"}, {"end": 163.339, "text": " And when we think about that title, you might have some scary thoughts, AI, eating a website.", "start": 156.014, "speaker": "SPEAKER_02"}, {"end": 164.78, "text": "What could that be?", "start": 164.16, "speaker": "SPEAKER_02"}, {"end": 166.361, "text": "That doesn't sound very friendly, right?", "start": 164.82, "speaker": "SPEAKER_02"}, {"end": 168.122, "text": "You might imagine some scary things.", "start": 166.381, "speaker": "SPEAKER_02"}, {"end": 171.384, "text": "Okay, so on the count of three, I'm gonna try to time this.", "start": 169.023, "speaker": "SPEAKER_02"}, {"end": 177.008, "text": "I want you to stomp a really scary stomp, and I'm gonna bring in a very scary AI thing.", "start": 171.424, "speaker": "SPEAKER_02"}, {"end": 178.449, "text": "Okay, you ready?", "start": 177.689, "speaker": "SPEAKER_02"}, {"end": 179.61, "text": "One, two, three.", "start": 178.469, "speaker": "SPEAKER_02"}, {"end": 185.242, "text": " Oh, you did it!", "start": 184.622, "speaker": "SPEAKER_02"}, {"end": 187.163, "text": "You conjured it!", "start": 185.442, "speaker": "SPEAKER_02"}, {"end": 192.045, "text": "So here's a giant computer brain eating up all these web pages.", "start": 188.104, "speaker": "SPEAKER_02"}, {"end": 192.265, "text": "Ah!", "start": 192.165, "speaker": "SPEAKER_02"}, {"end": 193.306, "text": "My little guys!", "start": 192.665, "speaker": "SPEAKER_02"}, {"end": 194.466, "text": "Let it go!", "start": 193.986, "speaker": "SPEAKER_02"}, {"end": 197.828, "text": "Anybody scared?", "start": 194.486, "speaker": "SPEAKER_02"}, {"end": 198.728, "text": "Okay, you can sit down.", "start": 198.008, "speaker": "SPEAKER_02"}, {"end": 203.37, "text": "Don't worry, all the little A guys, AI guys, we'll be back.", "start": 200.429, "speaker": "SPEAKER_02"}, {"end": 206.671, "text": "Woo!", "start": 206.291, "speaker": "SPEAKER_02"}, {"end": 207.512, "text": "Okay.", "start": 207.231, "speaker": "SPEAKER_02"}, {"end": 214.815, "text": " So in truth, when you think about AI, by the way, it's not as emphatic as when everybody pops a balloon.", "start": 209.033, "speaker": "SPEAKER_02"}, {"end": 215.356, "text": "But that was good.", "start": 214.835, "speaker": "SPEAKER_02"}, {"end": 215.916, "text": "I like that.", "start": 215.496, "speaker": "SPEAKER_02"}, {"end": 216.276, "text": "Thank you.", "start": 215.956, "speaker": "SPEAKER_02"}, {"end": 222.819, "text": "In truth, though, when most people think about AI, they think about something like chat GPT.", "start": 218.577, "speaker": "SPEAKER_02"}, {"end": 227.42, "text": "We saw a bunch of examples of people using prompt engineering through chat GPT.", "start": 222.859, "speaker": "SPEAKER_02"}, {"end": 229.681, "text": "This is a large language model.", "start": 227.44, "speaker": "SPEAKER_02"}, {"end": 231.662, "text": "You can ask it questions like, who am I?", "start": 229.741, "speaker": "SPEAKER_02"}, {"end": 232.302, "text": "Who is Luke?", "start": 231.802, "speaker": "SPEAKER_02"}, {"end": 237.985, "text": "So I don't really need an intro slide anymore, because all this stuff is just available.", "start": 233.223, "speaker": "SPEAKER_02"}, {"end": 246.629, "text": " But it's not just my website that these giant AI models have sort of subsumed and eaten up, right?", "start": 239.786, "speaker": "SPEAKER_02"}, {"end": 250.071, "text": "They've also eaten up the entire rest of the web.", "start": 246.669, "speaker": "SPEAKER_02"}, {"end": 253.473, "text": "And so when you ask a question like, where did Luke go to school?", "start": 250.572, "speaker": "SPEAKER_02"}, {"end": 257.075, "text": "It's going to respond to something that seems really likely, right?", "start": 254.294, "speaker": "SPEAKER_02"}, {"end": 260.797, "text": "Because people who do product designs very likely that they actually went to CMU.", "start": 257.095, "speaker": "SPEAKER_02"}, {"end": 262.118, "text": "Probably many of them did.", "start": 261.137, "speaker": "SPEAKER_02"}, {"end": 263.719, "text": "But I did not.", "start": 262.858, "speaker": "SPEAKER_02"}, {"end": 273.335, "text": " So I say, I think Luke went to UIC and it responds, you're right, I'm so sorry, but he went to grad school at CMU, which I did not.", "start": 265.669, "speaker": "SPEAKER_02"}, {"end": 277.278, "text": "So then I can say, did I really go to CMU?", "start": 274.115, "speaker": "SPEAKER_02"}, {"end": 278.739, "text": "You're like, of course not, you're right.", "start": 277.298, "speaker": "SPEAKER_02"}, {"end": 283.709, "text": " So no problem with reinforcement learning in these models.", "start": 280.948, "speaker": "SPEAKER_02"}, {"end": 287.891, "text": "They're very oriented to doing what you say, right?", "start": 283.749, "speaker": "SPEAKER_02"}, {"end": 288.631, "text": "You're wrong.", "start": 288.251, "speaker": "SPEAKER_02"}, {"end": 289.731, "text": "I apologize.", "start": 288.991, "speaker": "SPEAKER_02"}, {"end": 290.372, "text": "You're wrong again.", "start": 289.811, "speaker": "SPEAKER_02"}, {"end": 291.132, "text": "I apologize.", "start": 290.432, "speaker": "SPEAKER_02"}, {"end": 294.393, "text": "So we can kind of get them into whatever state we want.", "start": 291.492, "speaker": "SPEAKER_02"}, {"end": 299.956, "text": "And sometimes they'll veer off into weird places, which I'm not going to do because the intro to this talk was weird enough.", "start": 294.413, "speaker": "SPEAKER_02"}, {"end": 304.844, "text": " But let's zoom out for a second and look at this relationship.", "start": 302.403, "speaker": "SPEAKER_02"}, {"end": 311.826, "text": "So we have websites, like my little website, where probably that information about me existed or maybe existed on the SmashingConf website.", "start": 304.864, "speaker": "SPEAKER_02"}, {"end": 315.107, "text": "And you've got lots and lots of these websites, right?", "start": 312.586, "speaker": "SPEAKER_02"}, {"end": 321.229, "text": "And all these things get basically turned into training data for these immense models.", "start": 315.467, "speaker": "SPEAKER_02"}, {"end": 326.451, "text": "And they then can generate a response based on all the things they've learned about.", "start": 321.889, "speaker": "SPEAKER_02"}, {"end": 328.812, "text": "What's the most likely thing to come next?", "start": 326.491, "speaker": "SPEAKER_02"}, {"end": 334.474, "text": " And then they get tuned in different directions with things like reinforcement learning.", "start": 330.092, "speaker": "SPEAKER_02"}, {"end": 340.537, "text": "So hence why it assumes that I as a product designer went to CMU.", "start": 335.014, "speaker": "SPEAKER_02"}, {"end": 352.523, "text": "But this whole concept of training giant machine brains on the totality of published content on the internet", "start": 341.618, "speaker": "SPEAKER_02"}, {"end": 354.884, "text": " You know, that doesn't sound nice, right?", "start": 353.263, "speaker": "SPEAKER_02"}, {"end": 360.187, "text": "It sounds kind of scary, hence my scary AI thing here, conjures these scary narratives.", "start": 354.904, "speaker": "SPEAKER_02"}, {"end": 368.031, "text": "But it doesn't have to be this way, because if we stop thinking in terms of this monolithic giant AI brain,", "start": 360.867, "speaker": "SPEAKER_02"}, {"end": 374.379, "text": " And we instead start thinking about all the things different AI models, large or small, can do.", "start": 369.011, "speaker": "SPEAKER_02"}, {"end": 381.649, "text": "And then we start thinking about how they can actually help us create new user experiences, build products in different ways.", "start": 374.98, "speaker": "SPEAKER_02"}, {"end": 384.333, "text": "We heard a little bit about those in the morning sessions.", "start": 381.689, "speaker": "SPEAKER_02"}, {"end": 387.577, "text": " There's a lot of things these things can do with language.", "start": 385.356, "speaker": "SPEAKER_02"}, {"end": 390.877, "text": "They can basically take speech, turn into text.", "start": 388.297, "speaker": "SPEAKER_02"}, {"end": 393.638, "text": "They can take text, turn it into speech.", "start": 391.337, "speaker": "SPEAKER_02"}, {"end": 396.339, "text": "They can transform languages from one to the other.", "start": 393.738, "speaker": "SPEAKER_02"}, {"end": 400.14, "text": "With vision, they can identify objects, identify faces.", "start": 396.379, "speaker": "SPEAKER_02"}, {"end": 401.78, "text": "They can segment stuff.", "start": 400.6, "speaker": "SPEAKER_02"}, {"end": 408.822, "text": "There's this huge list of capabilities that we now have because of these very capable AI models.", "start": 402.22, "speaker": "SPEAKER_02"}, {"end": 414.323, "text": "And what's different from a few years ago, we have always had, not always, but machine learning's been around for a long time.", "start": 408.862, "speaker": "SPEAKER_02"}, {"end": 423.426, "text": " But the dramatic shift over the past maybe two-ish years is that these very, very capable models are now accessible to anyone, anywhere and everywhere.", "start": 414.983, "speaker": "SPEAKER_02"}, {"end": 426.387, "text": "You can download them, you can run them from the cloud.", "start": 423.826, "speaker": "SPEAKER_02"}, {"end": 432.63, "text": "And so all these capabilities are just flying all over the place and showing up in so many different places.", "start": 426.828, "speaker": "SPEAKER_02"}, {"end": 437.652, "text": "So people are learning about all these capabilities and integrating them into lots of different places.", "start": 433.09, "speaker": "SPEAKER_02"}, {"end": 444.041, "text": " And this is the way I really like to think about this change, as opposed to the scary AI brain.", "start": 438.737, "speaker": "SPEAKER_02"}, {"end": 454.127, "text": "To me, it's much more like, what are the new ingredients that we currently have that we can use to rethink experiences, to do things we couldn't do before, right?", "start": 444.681, "speaker": "SPEAKER_02"}, {"end": 460.952, "text": "To go from all of these different technologies to new capabilities, especially on the web.", "start": 454.508, "speaker": "SPEAKER_02"}, {"end": 465.835, "text": "Because God bless Brad and his enthusiasm for the original principles of the web.", "start": 461.492, "speaker": "SPEAKER_02"}, {"end": 468.157, "text": "The web is not a very cool place right now.", "start": 466.095, "speaker": "SPEAKER_02"}, {"end": 478.011, "text": " You go there and you're just inudated with like 15 dialogues and subscriptions for newsletters and you can't even read the content and honestly the first thing you want to do is kind of leave.", "start": 469.058, "speaker": "SPEAKER_02"}, {"end": 480.534, "text": "It's not very cool.", "start": 478.972, "speaker": "SPEAKER_02"}, {"end": 483.959, "text": "But I think we can make it really cool.", "start": 482.177, "speaker": "SPEAKER_02"}, {"end": 490.751, "text": " So with that in mind, I'm just gonna walk you through a whole bunch of details on putting these things to work.", "start": 484.986, "speaker": "SPEAKER_02"}, {"end": 498.158, "text": "And the place where I've been putting them to work is the place that I can control the most, which is my little part of the worldwide web here.", "start": 491.572, "speaker": "SPEAKER_02"}, {"end": 499.039, "text": "This is my website.", "start": 498.178, "speaker": "SPEAKER_02"}, {"end": 504.684, "text": "I started this 28 years ago, so how many of you are less than 28 years old?", "start": 499.799, "speaker": "SPEAKER_02"}, {"end": 507.166, "text": "Okay, yeah, that's what I thought.", "start": 505.945, "speaker": "SPEAKER_02"}, {"end": 513.533, "text": " What can you do in 28 years of having a website so you have things to aspire to?", "start": 509.451, "speaker": "SPEAKER_02"}, {"end": 522.298, "text": "Well, I've written 2,000 text articles, put up about 100-something videos, bunch of PDF presentations, made almost 2,000 images.", "start": 513.593, "speaker": "SPEAKER_02"}, {"end": 524.319, "text": "Yes, you too can achieve such things.", "start": 522.398, "speaker": "SPEAKER_02"}, {"end": 528.301, "text": "That's a lot of stuff.", "start": 527.481, "speaker": "SPEAKER_02"}, {"end": 531.863, "text": "And the way you can make use of that stuff", "start": 529.782, "speaker": "SPEAKER_02"}, {"end": 536.943, "text": " using a web browser or the internet today is mostly like two modalities.", "start": 533.599, "speaker": "SPEAKER_02"}, {"end": 538.185, "text": "You can search for things, right?", "start": 536.963, "speaker": "SPEAKER_02"}, {"end": 540.127, "text": "You come back with some results.", "start": 538.205, "speaker": "SPEAKER_02"}, {"end": 541.168, "text": "You can browse around.", "start": 540.167, "speaker": "SPEAKER_02"}, {"end": 547.355, "text": "But even just my little website has 500,000 written words, 6,000 minutes of video.", "start": 541.849, "speaker": "SPEAKER_02"}, {"end": 551.56, "text": "So like, God bless you if you ever make your way even through part of that.", "start": 548.176, "speaker": "SPEAKER_02"}, {"end": 553.34, "text": " Highly unlikely.", "start": 552.54, "speaker": "SPEAKER_02"}, {"end": 557.942, "text": "But what if we didn't have to just do search and browse?", "start": 554.501, "speaker": "SPEAKER_02"}, {"end": 559.822, "text": "What if we could actually mirror the real world?", "start": 557.982, "speaker": "SPEAKER_02"}, {"end": 564.444, "text": "And the real world that we're in, Vitaly keeps reminding us about the Pac-Man, right?", "start": 560.643, "speaker": "SPEAKER_02"}, {"end": 565.284, "text": "Talk to each other.", "start": 564.504, "speaker": "SPEAKER_02"}, {"end": 567.265, "text": "What if you could just talk?", "start": 566.024, "speaker": "SPEAKER_02"}, {"end": 574.067, "text": "And so you could ask me a question like, hey, Luke, how do I decide what metrics I should use for my app?", "start": 568.205, "speaker": "SPEAKER_02"}, {"end": 580.252, "text": " And in the real world, what I would do is be like, hmm, what do I know about that?", "start": 575.129, "speaker": "SPEAKER_02"}, {"end": 584.816, "text": "I'll think about maybe a bunch of the things I've written, maybe some of the talks I've given in the past.", "start": 580.273, "speaker": "SPEAKER_02"}, {"end": 590.139, "text": "And then I, in my head, assemble an answer and really quickly respond to you.", "start": 585.356, "speaker": "SPEAKER_02"}, {"end": 595.202, "text": "The human brain is an amazing thing, much more amazing than any of these AI models we have.", "start": 590.159, "speaker": "SPEAKER_02"}, {"end": 604.509, "text": "But I would think about what I know and within, you know, fractions of a second be able to assemble it into a coherent response and just keep talking and answer you.", "start": 595.242, "speaker": "SPEAKER_02"}, {"end": 607.691, "text": " That's pretty different than our options online.", "start": 605.149, "speaker": "SPEAKER_02"}, {"end": 611.354, "text": "If you do search, how do I pick metrics from my app?", "start": 608.452, "speaker": "SPEAKER_02"}, {"end": 613.396, "text": "Go fish through these links.", "start": 611.434, "speaker": "SPEAKER_02"}, {"end": 620.662, "text": "If you're browsing around and you make your way to all the articles tagged with metrics, well, there's 243 of them.", "start": 614.637, "speaker": "SPEAKER_02"}, {"end": 622.544, "text": "So again, good luck.", "start": 620.722, "speaker": "SPEAKER_02"}, {"end": 626.527, "text": "But what if instead we can do what I'm saying?", "start": 624.385, "speaker": "SPEAKER_02"}, {"end": 628.529, "text": "How do I pick metrics from my app?", "start": 626.747, "speaker": "SPEAKER_02"}, {"end": 637.743, "text": " So go through everything I've written or said on the topic, put them together into a coherent response, cite the sources, so you can go even deeper or verify what I said.", "start": 629.639, "speaker": "SPEAKER_02"}, {"end": 641.344, "text": "So in this case, this response, how do I pick metrics from my app?", "start": 638.623, "speaker": "SPEAKER_02"}, {"end": 645.186, "text": "You could see if you put this in the little system, I'm going to show you.", "start": 641.925, "speaker": "SPEAKER_02"}, {"end": 646.266, "text": "You get this answer.", "start": 645.386, "speaker": "SPEAKER_02"}, {"end": 650.188, "text": "And one of it is coming from a talk called Mind the Gap from 1 Minute 17.", "start": 646.827, "speaker": "SPEAKER_02"}, {"end": 671.697, "text": " one hour 17 minutes and 50 seconds you click on that and it will take you to that part of the video right there queued up and you can see on the slide in the background how do I decide what to measure so those citations are tied directly to where the information is coming from so", "start": 652.189, "speaker": "SPEAKER_02"}, {"end": 681.738, "text": " This is what happens when a bunch of AI models effectively eat all the pieces of content that make up a website and spit up a very different interaction model.", "start": 672.935, "speaker": "SPEAKER_02"}, {"end": 685.8, "text": "And the first question people always have is, well, how do you do that?", "start": 682.839, "speaker": "SPEAKER_02"}, {"end": 686.82, "text": "Well, I'll get to that.", "start": 685.9, "speaker": "SPEAKER_02"}, {"end": 691.922, "text": "But first, I want to look at what the experience is, and then I'll talk through the pieces of how it gets put together.", "start": 686.86, "speaker": "SPEAKER_02"}, {"end": 695.403, "text": "Now, when you see this, I ask a question, you respond with an answer.", "start": 692.502, "speaker": "SPEAKER_02"}, {"end": 697.804, "text": "Many people say, well, isn't this just a chatbot?", "start": 695.463, "speaker": "SPEAKER_02"}, {"end": 702.585, "text": " What LLM did you use for this chatbot?", "start": 699.783, "speaker": "SPEAKER_02"}, {"end": 703.706, "text": "Is this chat GPT?", "start": 702.765, "speaker": "SPEAKER_02"}, {"end": 706.228, "text": "This is what most people start thinking.", "start": 704.727, "speaker": "SPEAKER_02"}, {"end": 711.261, "text": " Why does chat keep showing up?", "start": 708.76, "speaker": "SPEAKER_02"}, {"end": 715.263, "text": "Number one, chat is a paradigm that's extremely familiar to all of us.", "start": 711.301, "speaker": "SPEAKER_02"}, {"end": 721.786, "text": "You all have probably spent way too much time in your messaging app, even today, some of you may be on it right now.", "start": 716.083, "speaker": "SPEAKER_02"}, {"end": 729.269, "text": "So this whole paradigm of messaging people, very familiar, very ubiquitous in our lives.", "start": 722.206, "speaker": "SPEAKER_02"}, {"end": 731.97, "text": "So it's quite easy to kind of get in and running with that.", "start": 729.309, "speaker": "SPEAKER_02"}, {"end": 735.412, "text": "But the pattern of this is a series of alternating messages, right?", "start": 732.01, "speaker": "SPEAKER_02"}, {"end": 738.113, "text": "I say something, someone replies, the list goes on,", "start": 735.492, "speaker": "SPEAKER_02"}, {"end": 742.957, "text": " If a message runs long, the space for it, you know, you get what these things look like, right?", "start": 738.593, "speaker": "SPEAKER_02"}, {"end": 751.284, "text": "But is that really a great interaction design pattern for communicating to a global super brain?", "start": 744.118, "speaker": "SPEAKER_02"}, {"end": 752.905, "text": "Maybe.", "start": 752.645, "speaker": "SPEAKER_02"}, {"end": 753.746, "text": "I don't know.", "start": 752.925, "speaker": "SPEAKER_02"}, {"end": 755.147, "text": "Probably not, though.", "start": 754.507, "speaker": "SPEAKER_02"}, {"end": 763.574, "text": "If you actually look and see some of the issues that come up with this type of pattern, and this is some research from Nielsen Norman,", "start": 756.468, "speaker": "SPEAKER_02"}, {"end": 769.103, "text": " You know, it's not really optimal for an iterative conversation with a set of documents.", "start": 764.38, "speaker": "SPEAKER_02"}, {"end": 773.687, "text": "So they did a bunch of tests on these large language model-based chat experiences.", "start": 769.524, "speaker": "SPEAKER_02"}, {"end": 782.873, "text": "And in particular, people having to kind of scroll through these conversation threads to find things, remember what was what, kind of veer things in the right direction.", "start": 774.347, "speaker": "SPEAKER_02"}, {"end": 790.779, "text": "And a quote on the bottom here I think is pretty meaningful, which is, this behavior is a significant point of friction for everybody.", "start": 782.893, "speaker": "SPEAKER_02"}, {"end": 792.96, "text": " that they observed in these studies.", "start": 791.559, "speaker": "SPEAKER_02"}, {"end": 794.661, "text": "So we could probably do a bit better.", "start": 793.18, "speaker": "SPEAKER_02"}, {"end": 806.63, "text": "Now, to account for some of the things that they found in those studies, we made use of a different interaction pattern when creating the experience you saw a second ago.", "start": 796.082, "speaker": "SPEAKER_02"}, {"end": 815.556, "text": "So instead of this back and forth chat, everything expands, what you'll find in this design is something that looks a little bit more like this.", "start": 806.65, "speaker": "SPEAKER_02"}, {"end": 819.218, "text": " And so what does it actually look like?", "start": 817.335, "speaker": "SPEAKER_02"}, {"end": 827.991, "text": "Basically, all the previous question and answer pairs get collapsed after you've messaged them or completed them.", "start": 819.538, "speaker": "SPEAKER_02"}, {"end": 832.458, "text": "Then the current one that you're on will open up, show you lots of space for the answer.", "start": 828.151, "speaker": "SPEAKER_02"}, {"end": 836.3, "text": " And you can kind of go back and forth within them.", "start": 833.599, "speaker": "SPEAKER_02"}, {"end": 838.702, "text": "So let me just show you what this looks like rather than talking about diagrams.", "start": 836.32, "speaker": "SPEAKER_02"}, {"end": 841.183, "text": "So here you can see, how does free shipping help?", "start": 838.722, "speaker": "SPEAKER_02"}, {"end": 842.643, "text": "It's part of this conversation.", "start": 841.263, "speaker": "SPEAKER_02"}, {"end": 844.024, "text": "There's the response.", "start": 843.124, "speaker": "SPEAKER_02"}, {"end": 849.426, "text": "If I want to go back and see how do you provide real time feedback, then I open up that one.", "start": 844.124, "speaker": "SPEAKER_02"}, {"end": 852.168, "text": "And how does free shipping help thing close up?", "start": 849.527, "speaker": "SPEAKER_02"}, {"end": 857.711, "text": " So I can kind of access one question and answer pair at a time.", "start": 853.068, "speaker": "SPEAKER_02"}, {"end": 864.255, "text": "And then I can focus on that specific question that I have and its response when I'm asking it.", "start": 858.331, "speaker": "SPEAKER_02"}, {"end": 867.236, "text": "And the rest kind of just rolls up and gets out of the way.", "start": 864.295, "speaker": "SPEAKER_02"}, {"end": 871.139, "text": "So you can much more easily scan, find the thing you were looking for before, and get back to it.", "start": 867.276, "speaker": "SPEAKER_02"}, {"end": 876.982, "text": "And this interaction actually works a little bit more like a FAQ style pattern.", "start": 872.419, "speaker": "SPEAKER_02"}, {"end": 878.263, "text": "You guys have probably seen these.", "start": 877.022, "speaker": "SPEAKER_02"}, {"end": 883.721, "text": " So it's like a big ask thing, right?", "start": 880.318, "speaker": "SPEAKER_02"}, {"end": 887.644, "text": "And then there's all these frequently asked questions that you can open up and expand.", "start": 883.741, "speaker": "SPEAKER_02"}, {"end": 889.626, "text": "Pretty much most websites have this stuff.", "start": 887.844, "speaker": "SPEAKER_02"}, {"end": 895.759, "text": " And so this is a nice example of when you start from the problem space, right?", "start": 890.956, "speaker": "SPEAKER_02"}, {"end": 897.561, "text": "When you don't start with, oh, we need a chatbot.", "start": 895.779, "speaker": "SPEAKER_02"}, {"end": 899.062, "text": "Let's implement a chatbot.", "start": 897.761, "speaker": "SPEAKER_02"}, {"end": 900.743, "text": "And you think about, well, what are we trying to do?", "start": 899.182, "speaker": "SPEAKER_02"}, {"end": 902.664, "text": "Well, people are asking questions.", "start": 901.263, "speaker": "SPEAKER_02"}, {"end": 904.285, "text": "They're trying to get responses.", "start": 902.704, "speaker": "SPEAKER_02"}, {"end": 907.207, "text": "It's tied to a specific set of information.", "start": 904.345, "speaker": "SPEAKER_02"}, {"end": 911.23, "text": "Maybe there's a different interaction paradigm that makes a little bit more sense here.", "start": 907.607, "speaker": "SPEAKER_02"}, {"end": 917.454, "text": "And this Q&A interface that I just showed you looks a little bit more like a FAQ pattern than a chatbot pattern.", "start": 912.31, "speaker": "SPEAKER_02"}, {"end": 925.802, "text": " And I bring this up because too often the process of design starts with something like a competitive audit.", "start": 918.7, "speaker": "SPEAKER_02"}, {"end": 929.843, "text": "Or let's go see what everybody else is doing, put it up on a wall, right?", "start": 926.322, "speaker": "SPEAKER_02"}, {"end": 936.625, "text": "And then as soon as you do that, if you've ever ridden a bike and you're riding a bike and you look at a tree, where are you gonna go?", "start": 930.543, "speaker": "SPEAKER_02"}, {"end": 939.426, "text": "You're gonna ride right into the tree, right?", "start": 937.685, "speaker": "SPEAKER_02"}, {"end": 941.306, "text": "It's called target fixation.", "start": 940.006, "speaker": "SPEAKER_02"}, {"end": 945.387, "text": "So as soon as you see that chatbot pattern on chatGPT, guess where your design is gonna go?", "start": 941.326, "speaker": "SPEAKER_02"}, {"end": 947.388, "text": "Right into that pattern.", "start": 946.448, "speaker": "SPEAKER_02"}, {"end": 959.887, "text": " And whether intentionally or not, you end up copying what these different things are doing instead of letting the problem space actually guide the solution.", "start": 948.641, "speaker": "SPEAKER_02"}, {"end": 975.056, "text": "So in this case, starting with understanding that problem of people are asking questions, they need answers, there's going to be a lot of them, they're going to go back and try and reference things, they need a way to scan through stuff, we ended up a little bit more of a FAQ thing than a chatbot thing.", "start": 960.788, "speaker": "SPEAKER_02"}, {"end": 980.247, "text": " So now there's this expandable conversational interface.", "start": 976.485, "speaker": "SPEAKER_02"}, {"end": 984.208, "text": "You ask a question, response comes down, some citations come up.", "start": 980.267, "speaker": "SPEAKER_02"}, {"end": 990.731, "text": "But for the purpose of this presentation, because these are big slides on a screen, I've simplified a few things here.", "start": 985.149, "speaker": "SPEAKER_02"}, {"end": 994.673, "text": "So I actually want to show you two more things on how this interface works.", "start": 990.771, "speaker": "SPEAKER_02"}, {"end": 999.055, "text": "And one important one is this concept of suggested content.", "start": 995.533, "speaker": "SPEAKER_02"}, {"end": 1001.816, "text": "So you could see here.", "start": 1000.395, "speaker": "SPEAKER_02"}, {"end": 1010.125, "text": " there's a response and then immediately after the response there's a series of follow-on questions that are related to the question you just asked.", "start": 1003.24, "speaker": "SPEAKER_02"}, {"end": 1019.951, "text": "The first couple are directly tied to some of the questions that show up there and then the other ones are broader and kind of expand to other things that you can ask.", "start": 1010.745, "speaker": "SPEAKER_02"}, {"end": 1026.496, "text": "And these suggested questions are actually really critical in anything that is just this big open-ended", "start": 1021.192, "speaker": "SPEAKER_02"}, {"end": 1028.075, "text": " Box.", "start": 1027.694, "speaker": "SPEAKER_02"}, {"end": 1031.74, "text": "Because one of the big problems people have with these things is, what do I ask?", "start": 1028.876, "speaker": "SPEAKER_02"}, {"end": 1032.201, "text": "I don't know.", "start": 1031.84, "speaker": "SPEAKER_02"}, {"end": 1034.083, "text": "Well, you can ask anything.", "start": 1033.102, "speaker": "SPEAKER_02"}, {"end": 1036.627, "text": "That's like saying, don't ask anything.", "start": 1034.905, "speaker": "SPEAKER_02"}, {"end": 1037.749, "text": "Nobody knows.", "start": 1037.188, "speaker": "SPEAKER_02"}, {"end": 1041.559, "text": " And so having something like this addresses the issue of what should I ask.", "start": 1038.737, "speaker": "SPEAKER_02"}, {"end": 1053.588, "text": "And lots of people have that problem because something like 65 or 70% of all the questions that get asked in this interface come from these suggested questions, right?", "start": 1041.599, "speaker": "SPEAKER_02"}, {"end": 1057.23, "text": "So they're very helpful for getting people started and moving them in the right direction.", "start": 1053.688, "speaker": "SPEAKER_02"}, {"end": 1065.556, "text": "So we knew from the start these were important, but it took a bit to actually iterate through the designs to get to what you saw there.", "start": 1058.191, "speaker": "SPEAKER_02"}, {"end": 1072.284, "text": " So at first we experimented with something like very explicit to trigger suggested questions, because we're like, oh, we don't want to show these all the time.", "start": 1066.477, "speaker": "SPEAKER_02"}, {"end": 1076.45, "text": "Somebody should be able to kind of click on something and get those.", "start": 1072.344, "speaker": "SPEAKER_02"}, {"end": 1078.072, "text": "So we started with this little light bulb, right?", "start": 1076.47, "speaker": "SPEAKER_02"}, {"end": 1079.293, "text": "Give me an idea, get it?", "start": 1078.352, "speaker": "SPEAKER_02"}, {"end": 1083.478, "text": "So you click the light bulb, bunch of suggested questions come up.", "start": 1079.373, "speaker": "SPEAKER_02"}, {"end": 1088.579, "text": " But then you get the classic, nobody knows what that icon is, are they gonna click it?", "start": 1084.517, "speaker": "SPEAKER_02"}, {"end": 1091.481, "text": "So then you try something like, what else can I ask?", "start": 1088.619, "speaker": "SPEAKER_02"}, {"end": 1092.381, "text": "And that shows up.", "start": 1091.541, "speaker": "SPEAKER_02"}, {"end": 1097.684, "text": "But in both cases, the burden is on people to ask for these things.", "start": 1093.422, "speaker": "SPEAKER_02"}, {"end": 1107.709, "text": "So instead, here's a series of designs that actually just started dropping it right after every answer, you'd get these suggestions.", "start": 1098.624, "speaker": "SPEAKER_02"}, {"end": 1111.711, "text": "And there was a couple iterations on this until finally,", "start": 1108.369, "speaker": "SPEAKER_02"}, {"end": 1118.268, "text": " We ended up on the most obvious way you could possibly do this, which is the way design always works.", "start": 1113.464, "speaker": "SPEAKER_02"}, {"end": 1119.569, "text": "It's obvious in retrospect.", "start": 1118.348, "speaker": "SPEAKER_02"}, {"end": 1122.791, "text": "Just put it directly in line right after the answer.", "start": 1119.989, "speaker": "SPEAKER_02"}, {"end": 1126.975, "text": "Make it look almost like the answers, except a tiny bit different and away you go.", "start": 1122.872, "speaker": "SPEAKER_02"}, {"end": 1133.88, "text": "So these follow directly in line in the interface.", "start": 1129.277, "speaker": "SPEAKER_02"}, {"end": 1141.707, "text": "And the other important detail that I'm not showing, which is something we just launched a week ago, maybe two weeks ago,", "start": 1134.761, "speaker": "SPEAKER_02"}, {"end": 1149.963, "text": " is now we created this new capability and you have to like squint the CDC is little like ones and twos popping up over there.", "start": 1142.894, "speaker": "SPEAKER_02"}, {"end": 1160.216, "text": "These now are direct citations of the part of the answer that relates to where that content came from.", "start": 1151.024, "speaker": "SPEAKER_02"}, {"end": 1165.771, "text": " So you can kind of see within the response which information piece is exciting.", "start": 1161.468, "speaker": "SPEAKER_02"}, {"end": 1170.555, "text": "And if you click on them, as I'll show you in a second, you can go and see that information.", "start": 1165.951, "speaker": "SPEAKER_02"}, {"end": 1175.579, "text": "So further kind of making obvious the way this thing works.", "start": 1171.556, "speaker": "SPEAKER_02"}, {"end": 1180.582, "text": "And the way this works is it's pulling all this information from my website, right?", "start": 1176.399, "speaker": "SPEAKER_02"}, {"end": 1187.948, "text": "All those articles, all those videos, all those audio files, and using that information to create a bunch of responses.", "start": 1180.642, "speaker": "SPEAKER_02"}, {"end": 1191.251, "text": " for people's questions about design.", "start": 1189.71, "speaker": "SPEAKER_02"}, {"end": 1198.016, "text": "Now, if you see one of those little citations, you can click on them and it'll expand it directly in line.", "start": 1192.272, "speaker": "SPEAKER_02"}, {"end": 1202.198, "text": "So to go back to my diagram before here, how does free shipping help?", "start": 1198.216, "speaker": "SPEAKER_02"}, {"end": 1208.502, "text": "You can see it's saying this article on evolving e-commerce shipping was used to make this answer.", "start": 1202.418, "speaker": "SPEAKER_02"}, {"end": 1215.687, "text": "You can open up and there's the article and you have the ability to ask questions just of that article or go back to this more global mode.", "start": 1208.562, "speaker": "SPEAKER_02"}, {"end": 1221.319, "text": " So what kinds of things can answer questions is not just those articles.", "start": 1217.915, "speaker": "SPEAKER_02"}, {"end": 1225.324, "text": "As I mentioned before, there's videos, podcasts, PDFs, images, and more.", "start": 1221.539, "speaker": "SPEAKER_02"}, {"end": 1231.572, "text": "But how do you get this stuff to be usable in this kind of interface?", "start": 1226.766, "speaker": "SPEAKER_02"}, {"end": 1238.479, "text": " You have web pages, they have PDFs, you have websites, they've got images, you've got all this stuff.", "start": 1233.956, "speaker": "SPEAKER_02"}, {"end": 1243.422, "text": "How do we take those things and then use them in this type of interface?", "start": 1238.519, "speaker": "SPEAKER_02"}, {"end": 1254.009, "text": "So there's a couple things that we need to do and this is where the ability for you to use a whole bunch of different AI models starts to, I think, emerge.", "start": 1244.223, "speaker": "SPEAKER_02"}, {"end": 1259.471, "text": " So, for example, if we look at videos, we have like a video, like this talk, let's say, is recorded.", "start": 1254.549, "speaker": "SPEAKER_02"}, {"end": 1261.371, "text": "There'll be a video file at the end of it.", "start": 1259.791, "speaker": "SPEAKER_02"}, {"end": 1267.473, "text": "We can use a speech-to-text model to take that video file, turn it into text.", "start": 1261.871, "speaker": "SPEAKER_02"}, {"end": 1273.015, "text": "We can use a diarization model to break up who's talking when, when there's the Q&A session.", "start": 1268.013, "speaker": "SPEAKER_02"}, {"end": 1275.916, "text": "We can have different voices and split that up.", "start": 1273.575, "speaker": "SPEAKER_02"}, {"end": 1285.668, "text": " We can generate summaries, keywords, and then we can use vision models to extract items in those video screenshots, people, that kind of thing.", "start": 1276.696, "speaker": "SPEAKER_02"}, {"end": 1295.456, "text": " So we take these files, we run them through a bunch of AI tasks using a number of models, and then we have to do a bunch of cleanup of that content.", "start": 1286.673, "speaker": "SPEAKER_02"}, {"end": 1301.778, "text": "And when I say cleanup, I mean like we make sure that it understands that Luke is Luke, right?", "start": 1296.256, "speaker": "SPEAKER_02"}, {"end": 1303.819, "text": "So it kind of cleans up the entities.", "start": 1301.998, "speaker": "SPEAKER_02"}, {"end": 1308.14, "text": "We add some metadata, like date, location, et cetera.", "start": 1303.859, "speaker": "SPEAKER_02"}, {"end": 1309.961, "text": "And so now we have", "start": 1308.86, "speaker": "SPEAKER_02"}, {"end": 1316.203, "text": " content that we can use to start answering people's questions that before was just a bunch of video files.", "start": 1311.141, "speaker": "SPEAKER_02"}, {"end": 1322.005, "text": "And to show you how far you can go with this, I'll give you one example.", "start": 1317.383, "speaker": "SPEAKER_02"}, {"end": 1327.186, "text": "One of the things that we wanted to do was actually get some nice looking thumbnails for these videos.", "start": 1322.065, "speaker": "SPEAKER_02"}, {"end": 1329.467, "text": "As you can see here, they're kind of all over the place.", "start": 1327.527, "speaker": "SPEAKER_02"}, {"end": 1336.169, "text": "And so what you can do pretty quickly is generate a bunch of thumbnails from a video.", "start": 1330.408, "speaker": "SPEAKER_02"}, {"end": 1339.611, "text": "This is a pretty trivial script with something like FFMpeg or what have you.", "start": 1336.189, "speaker": "SPEAKER_02"}, {"end": 1351.576, "text": " spit out a bunch of videos, and then I can manually just go through four or five of those videos, or up to ten or something like that, and pick a good looking thumbnail that I like.", "start": 1340.951, "speaker": "SPEAKER_02"}, {"end": 1363.762, "text": "And then using that little set of data, here's a bunch of thumbnails, pick this one, here's a bunch of thumbnails, pick this one, as examples, we can fine tune a little AI model that for all the rest of the hundred something videos picks a thumbnail for us.", "start": 1352.437, "speaker": "SPEAKER_02"}, {"end": 1375.545, "text": " So the next time it goes through here, it's going to use this lightweight training that we give to pick this beautiful image of me here with the Which Paris Girl Are You screenshot.", "start": 1365.031, "speaker": "SPEAKER_02"}, {"end": 1381.751, "text": " It nailed that part.", "start": 1380.911, "speaker": "SPEAKER_02"}, {"end": 1383.632, "text": "It left me with a dumb little grin, but whatever.", "start": 1381.771, "speaker": "SPEAKER_02"}, {"end": 1385.992, "text": "It's probably the best thumbnail of the bunch.", "start": 1384.312, "speaker": "SPEAKER_02"}, {"end": 1391.893, "text": "So this is just a quick example of how you can think about having AI models do a lot of things for you.", "start": 1386.412, "speaker": "SPEAKER_02"}, {"end": 1401.835, "text": "If you can kind of model a behavior to use that word intentionally, and if you can show a machine how to do something, you can train that machine to start doing it.", "start": 1392.573, "speaker": "SPEAKER_02"}, {"end": 1405.616, "text": "And that, I think, is a really big switch to go off in your head.", "start": 1402.435, "speaker": "SPEAKER_02"}, {"end": 1414.799, "text": " All right, if I can show a machine how to do this, I can get the machine to do it pretty consistently, even little things like this.", "start": 1407.213, "speaker": "SPEAKER_02"}, {"end": 1424.026, "text": "And so this opens up a lot of possibilities in terms of, again, how you can invert experiences because you can have machines do all this processing.", "start": 1416.42, "speaker": "SPEAKER_02"}, {"end": 1425.607, "text": "For audio, we can do the same thing.", "start": 1424.046, "speaker": "SPEAKER_02"}, {"end": 1433.013, "text": "Take a bunch of podcasts, take a bunch of audio interviews, turn them into text, diarize them, summarize them.", "start": 1426.147, "speaker": "SPEAKER_02"}, {"end": 1437.289, "text": " Whatever you need, do a bit of processing to them.", "start": 1434.508, "speaker": "SPEAKER_02"}, {"end": 1441.81, "text": "Again, same kind of thing for articles.", "start": 1439.329, "speaker": "SPEAKER_02"}, {"end": 1447.231, "text": "Summarize them, extract keywords, generate suggested questions, all this good stuff.", "start": 1442.89, "speaker": "SPEAKER_02"}, {"end": 1455.933, "text": "So we start with all of these files that make up your typical web page is, websites, audio, video, text.", "start": 1447.711, "speaker": "SPEAKER_02"}, {"end": 1458.194, "text": " images, what have you.", "start": 1457.333, "speaker": "SPEAKER_02"}, {"end": 1463.62, "text": "We use these different AI models to turn them into useful, common kinds of content.", "start": 1458.755, "speaker": "SPEAKER_02"}, {"end": 1468.845, "text": "And then when we have everything in a common format, this is where some of the secret sauce happens.", "start": 1464.04, "speaker": "SPEAKER_02"}, {"end": 1476.374, "text": "What we do is we take all those little bits of content, call them chunks, whatever you want to call them, and we basically put them into an embedding space.", "start": 1468.886, "speaker": "SPEAKER_02"}, {"end": 1478.856, "text": " that puts related concepts to it.", "start": 1476.974, "speaker": "SPEAKER_02"}, {"end": 1481.779, "text": "This isn't exactly how it works, but this is a good mental model to have.", "start": 1478.876, "speaker": "SPEAKER_02"}, {"end": 1484.522, "text": "So, e-commerce-y things go over here.", "start": 1481.839, "speaker": "SPEAKER_02"}, {"end": 1486.885, "text": "Mobile touchy things are over here.", "start": 1485.023, "speaker": "SPEAKER_02"}, {"end": 1487.566, "text": "What have you?", "start": 1487.105, "speaker": "SPEAKER_02"}, {"end": 1489.968, "text": "And in some cases, those are video pieces.", "start": 1487.586, "speaker": "SPEAKER_02"}, {"end": 1491.25, "text": "In other cases, those are images.", "start": 1489.988, "speaker": "SPEAKER_02"}, {"end": 1493.012, "text": "In other cases, those are text snippets.", "start": 1491.31, "speaker": "SPEAKER_02"}, {"end": 1510.79, "text": " So all this stuff is in basically one big, you can think of it as a fuzzy search index or another way to think about it as a semantic search index where you're actually looking things up by their relative meaning as opposed to things like keywords or metadata look up.", "start": 1495.374, "speaker": "SPEAKER_02"}, {"end": 1517.683, "text": " So I'll show a system diagram super quick just so folks get it like this always happens.", "start": 1513.14, "speaker": "SPEAKER_02"}, {"end": 1527.729, "text": "But you start with your docs, you do a whole bunch of this cleaning, you stick them into this search index that organizes them again in a very fuzzy kind of semantic way.", "start": 1517.723, "speaker": "SPEAKER_02"}, {"end": 1530.39, "text": "And then you need some way of searching against it.", "start": 1528.169, "speaker": "SPEAKER_02"}, {"end": 1533.372, "text": "You can have multiple ways of searching against it.", "start": 1531.451, "speaker": "SPEAKER_02"}, {"end": 1535.851, "text": " Back comes some results.", "start": 1534.611, "speaker": "SPEAKER_02"}, {"end": 1537.112, "text": "You can rank those results.", "start": 1535.911, "speaker": "SPEAKER_02"}, {"end": 1541.613, "text": "And then you take what you found and you basically use them to generate a response.", "start": 1537.712, "speaker": "SPEAKER_02"}, {"end": 1544.413, "text": "You can use lots of different models if you want.", "start": 1542.393, "speaker": "SPEAKER_02"}, {"end": 1545.473, "text": "You can use one model.", "start": 1544.453, "speaker": "SPEAKER_02"}, {"end": 1551.035, "text": "You can come up with a number of answers, pick the best one, using another AI model if you get the idea.", "start": 1546.073, "speaker": "SPEAKER_02"}, {"end": 1557.156, "text": "And then you get to a point where you've got the right information to answer people's questions.", "start": 1551.695, "speaker": "SPEAKER_02"}, {"end": 1559.765, "text": " So we break all the things into little chunks.", "start": 1557.864, "speaker": "SPEAKER_02"}, {"end": 1563.327, "text": "We stick them into this crazy vector space.", "start": 1560.225, "speaker": "SPEAKER_02"}, {"end": 1569.23, "text": "And then when people ask questions, we go find the relevant pieces and assemble them into answers.", "start": 1564.087, "speaker": "SPEAKER_02"}, {"end": 1572.932, "text": "It's AI models up and down.", "start": 1570.351, "speaker": "SPEAKER_02"}, {"end": 1574.633, "text": "In fact, it goes even deeper.", "start": 1573.412, "speaker": "SPEAKER_02"}, {"end": 1578.335, "text": "And hopefully, this is one thing that you take away from this.", "start": 1574.873, "speaker": "SPEAKER_02"}, {"end": 1583.558, "text": "If you go down the rabbit hole, there's like an AI model everywhere in this kind of workflow.", "start": 1578.375, "speaker": "SPEAKER_02"}, {"end": 1587.02, "text": " Because one of the important things to do is actually learn, right?", "start": 1584.158, "speaker": "SPEAKER_02"}, {"end": 1589.982, "text": "You get somebody an answer, you want to make sure that your system gets smarter.", "start": 1587.06, "speaker": "SPEAKER_02"}, {"end": 1605.973, "text": "So one of the things that we did a couple weeks ago is we took all of these lists of questions people ask, and then the list of corresponding documents that we used to answer them, and we trained a custom retriever.", "start": 1591.683, "speaker": "SPEAKER_02"}, {"end": 1611.667, "text": " which then spits out a score for how good is this document in answering the specific question.", "start": 1607.045, "speaker": "SPEAKER_02"}, {"end": 1614.629, "text": "And we did that with using that feedback that we were getting.", "start": 1612.027, "speaker": "SPEAKER_02"}, {"end": 1617.77, "text": "And if you may say, well, why does that matter?", "start": 1615.609, "speaker": "SPEAKER_02"}, {"end": 1633.778, "text": "If you just take some off-the-shelf AI model that sticks things into this embedding space, that kind of puts things in the vector space that you can later find it, those are just trained on general sets of sentences, like global internet stuff.", "start": 1618.13, "speaker": "SPEAKER_02"}, {"end": 1635.419, "text": "So they'll be okay.", "start": 1634.358, "speaker": "SPEAKER_02"}, {"end": 1639.796, "text": " at finding the pieces that are relevant in my content.", "start": 1636.753, "speaker": "SPEAKER_02"}, {"end": 1641.518, "text": "But they're not going to be great.", "start": 1640.457, "speaker": "SPEAKER_02"}, {"end": 1646.723, "text": "And if you make a custom one, these scores mean nothing to you except bigger is better.", "start": 1642.599, "speaker": "SPEAKER_02"}, {"end": 1654.731, "text": "So that cross-encoder compared to just using an open AI model does a heck of a lot better.", "start": 1647.383, "speaker": "SPEAKER_02"}, {"end": 1657.053, "text": "It's much, much better.", "start": 1655.832, "speaker": "SPEAKER_02"}, {"end": 1670.781, "text": " So you can make your own little AI models that find relevant bits of content, that understand what kind of content you have, and what good content is for the specific kinds of questions that get asked on your sites.", "start": 1658.891, "speaker": "SPEAKER_02"}, {"end": 1673.764, "text": "And the whole thing becomes a virtuous loop.", "start": 1671.682, "speaker": "SPEAKER_02"}, {"end": 1678.968, "text": "So some of you may be thinking, oh my god, that's a lot of work.", "start": 1674.724, "speaker": "SPEAKER_02"}, {"end": 1680.089, "text": "Why would we do this?", "start": 1679.188, "speaker": "SPEAKER_02"}, {"end": 1682.471, "text": "Why wouldn't I just show GPT on my website?", "start": 1680.149, "speaker": "SPEAKER_02"}, {"end": 1691.439, "text": " Well, if you ask ChadGPT a question like, why do so many designs look the same?", "start": 1686.755, "speaker": "SPEAKER_02"}, {"end": 1695.622, "text": "You're going to get this generic world knowledge answer, right?", "start": 1691.459, "speaker": "SPEAKER_02"}, {"end": 1704.049, "text": "It's going to be like, well, it's because of cognitive biases, legal and regulatory constraints, economic factors, the influence of trends and cultural influences.", "start": 1695.782, "speaker": "SPEAKER_02"}, {"end": 1706.932, "text": "It just sounds like a lot of horse shit to me.", "start": 1705.511, "speaker": "SPEAKER_02"}, {"end": 1715.7, "text": " So all that work I showed you actually allows us to deliver something that mirrors the way I would actually answer a question.", "start": 1709.713, "speaker": "SPEAKER_02"}, {"end": 1718.283, "text": "Kind of like reflects my specific opinions of things.", "start": 1715.76, "speaker": "SPEAKER_02"}, {"end": 1723.83, "text": "So instead, when you ask our interface, why do you so many designs look the same?", "start": 1718.844, "speaker": "SPEAKER_02"}, {"end": 1729.937, "text": "I basically say, look, it's because designers are just looking what everybody else is doing and kind of copying it without thinking too much.", "start": 1723.89, "speaker": "SPEAKER_02"}, {"end": 1733.415, "text": " very different kind of answer.", "start": 1731.934, "speaker": "SPEAKER_02"}, {"end": 1750.548, "text": "And then the other thing you get is all these citations that I was highlighting earlier, where again, you've got a bunch of information you've published in whatever websites you have or whatever applications you have, you can reference that stuff, make sure that it's cited and allow people to go deeper into all of it.", "start": 1734.336, "speaker": "SPEAKER_02"}, {"end": 1757.578, "text": " Now, when I say go deeper, it's worth showing a little bit of example of what that actually means.", "start": 1752.133, "speaker": "SPEAKER_02"}, {"end": 1760.781, "text": "So we asked this question, why do so many designs look the same?", "start": 1757.618, "speaker": "SPEAKER_02"}, {"end": 1768.107, "text": "And that answer comes from, I mentioned a few times, videos, audio, web pages, whatever.", "start": 1762.823, "speaker": "SPEAKER_02"}, {"end": 1776.734, "text": " If you were to click on one of those videos, what we're going to actually do is inside this conversational interface, bring up that video.", "start": 1769.188, "speaker": "SPEAKER_02"}, {"end": 1778.215, "text": "I showed you an example earlier.", "start": 1776.854, "speaker": "SPEAKER_02"}, {"end": 1787.842, "text": "Take you to that part of the video where that answer is showing up inside of the video track, and then give you the ability to ask questions of it, give you a.", "start": 1778.595, "speaker": "SPEAKER_02"}, {"end": 1797.714, "text": " Summary of that video give you a full transcript that you can search against, so you can click that transcript thing and look for whatever you want inside that video file.", "start": 1789.449, "speaker": "SPEAKER_02"}, {"end": 1810.181, "text": "If it's an audio file, same thing, we'll just embed an audio player, give you again that transcript and summary, allow you to ask questions of these multi-hour long audio files right then and there.", "start": 1798.895, "speaker": "SPEAKER_02"}, {"end": 1814.664, "text": "If it is an article, we'll put the article in there.", "start": 1811.582, "speaker": "SPEAKER_02"}, {"end": 1818.564, "text": " If it's a PDF, same thing, you kind of get it.", "start": 1816.303, "speaker": "SPEAKER_02"}, {"end": 1826.628, "text": "And so again, this is an example of how this interface, this conversational interface, has basically eaten all the stuff that previously made a website.", "start": 1819.004, "speaker": "SPEAKER_02"}, {"end": 1831.17, "text": "Because now all these things appear in the conversation.", "start": 1827.689, "speaker": "SPEAKER_02"}, {"end": 1831.971, "text": "You ask a question.", "start": 1831.19, "speaker": "SPEAKER_02"}, {"end": 1833.231, "text": "Here's the video that answers it.", "start": 1832.011, "speaker": "SPEAKER_02"}, {"end": 1834.512, "text": "Now you're watching that video.", "start": 1833.291, "speaker": "SPEAKER_02"}, {"end": 1835.853, "text": "You ask that a question.", "start": 1834.972, "speaker": "SPEAKER_02"}, {"end": 1836.733, "text": "Here's an audio file.", "start": 1835.913, "speaker": "SPEAKER_02"}, {"end": 1840.395, "text": "Now you can be inside that audio file or inside that article or through a PDF.", "start": 1836.773, "speaker": "SPEAKER_02"}, {"end": 1851.405, "text": " Now I said the word PDF and my body sort of cringed a little bit, because PDFs are special, as in very painfully special.", "start": 1842.182, "speaker": "SPEAKER_02"}, {"end": 1864.03, "text": "When I was building this interface, we first started with blog articles, and then we added video, and then we added audio, and then we added PDF.", "start": 1854.626, "speaker": "SPEAKER_02"}, {"end": 1872.041, "text": " And when we added something like 400-ish PDFs to here, you could see something changed pretty dramatically.", "start": 1865.174, "speaker": "SPEAKER_02"}, {"end": 1874.564, "text": "So before the question you could ask, what is design?", "start": 1872.081, "speaker": "SPEAKER_02"}, {"end": 1877.647, "text": "It would say something like, design is the art of gradually applying constraints.", "start": 1874.584, "speaker": "SPEAKER_02"}, {"end": 1879.75, "text": "So I kind of like this answer over here.", "start": 1877.948, "speaker": "SPEAKER_02"}, {"end": 1884.595, "text": "And then I added the 400 PDFs, and all of a sudden,", "start": 1880.49, "speaker": "SPEAKER_02"}, {"end": 1892.687, "text": " This one PDF file from 2006 about dogma free design like just dominated the results.", "start": 1886.405, "speaker": "SPEAKER_02"}, {"end": 1899.989, "text": "And now it says it is directed towards human beings with the aim of solving design is the fundamental soul of human creation.", "start": 1893.327, "speaker": "SPEAKER_02"}, {"end": 1901.009, "text": "I'm like, what is this?", "start": 1900.009, "speaker": "SPEAKER_02"}, {"end": 1902.57, "text": "This doesn't sound like me.", "start": 1901.029, "speaker": "SPEAKER_02"}, {"end": 1913.962, "text": " Oh, what turned out is inside this PDF, there was like four screens of quotes from other designers about what is design.", "start": 1905.211, "speaker": "SPEAKER_02"}, {"end": 1919.889, "text": "And so the model, the information retrieval model is like, hell yeah, look at this baby, right?", "start": 1914.803, "speaker": "SPEAKER_02"}, {"end": 1920.29, "text": "And it's like.", "start": 1919.929, "speaker": "SPEAKER_02"}, {"end": 1927.913, "text": " And so the answer just becomes like this regurgitation of a whole bunch of quotes about what is design.", "start": 1922.27, "speaker": "SPEAKER_02"}, {"end": 1930.314, "text": "So that's not good, right?", "start": 1929.073, "speaker": "SPEAKER_02"}, {"end": 1939.118, "text": "And the other thing that was happening was a lot of junk is kind of showing up as part of this content.", "start": 1931.534, "speaker": "SPEAKER_02"}, {"end": 1943.399, "text": " So if you break down what's inside, a PDF is like markup language, right?", "start": 1939.838, "speaker": "SPEAKER_02"}, {"end": 1947.921, "text": "It's got so much stuff in there about how things should look as opposed to what the content is.", "start": 1943.439, "speaker": "SPEAKER_02"}, {"end": 1951.742, "text": "And so you just get pieces of content that are just crap.", "start": 1948.741, "speaker": "SPEAKER_02"}, {"end": 1955.223, "text": "I mean, look at some of these things.", "start": 1951.822, "speaker": "SPEAKER_02"}, {"end": 1957.684, "text": "1,268, the content is zero nature.", "start": 1955.243, "speaker": "SPEAKER_02"}, {"end": 1965.167, "text": "One, two, six, five, biography photo, quote, auto video in the press.", "start": 1961.145, "speaker": "SPEAKER_02"}, {"end": 1971.349, "text": " So you're like, dear AI model, please answer the question with this.", "start": 1968.547, "speaker": "SPEAKER_02"}, {"end": 1975.192, "text": "And surprising the thing works at all.", "start": 1971.67, "speaker": "SPEAKER_02"}, {"end": 1977.674, "text": "It's actually a testament to how amazing these things that it works at all.", "start": 1975.212, "speaker": "SPEAKER_02"}, {"end": 1988.083, "text": "So what we ended up doing is cleaning this up again, getting rid of a bunch of stuff that's crap and clogging up our answers, introducing a mechanism where we diversify the answer.", "start": 1978.055, "speaker": "SPEAKER_02"}, {"end": 1992.426, "text": "So like, yo, yo, yo, you can't just use this one PDF even though it loves to have design quotes.", "start": 1988.123, "speaker": "SPEAKER_02"}, {"end": 1993.828, "text": "Break it up a little bit.", "start": 1992.887, "speaker": "SPEAKER_02"}, {"end": 1999.35, "text": " And when we did that, we actually got to this really nice balance answer for what is design.", "start": 1994.568, "speaker": "SPEAKER_02"}, {"end": 2004.191, "text": "There's a PDF there, there's a video, there's two podcasts, there's a text file.", "start": 2000.11, "speaker": "SPEAKER_02"}, {"end": 2009.513, "text": "And now it's starting to really represent the diversity of content and clean it up.", "start": 2005.772, "speaker": "SPEAKER_02"}, {"end": 2017.047, "text": " So the answer got a heck of a lot better through cleaning up all this content and balancing what shows up when.", "start": 2011.823, "speaker": "SPEAKER_02"}, {"end": 2019.85, "text": "And hopefully I'm getting through to you.", "start": 2018.288, "speaker": "SPEAKER_02"}, {"end": 2022.392, "text": "You know, people ask, like, what chatbot did you use for this?", "start": 2019.87, "speaker": "SPEAKER_02"}, {"end": 2023.793, "text": "What LLM is this?", "start": 2022.852, "speaker": "SPEAKER_02"}, {"end": 2024.874, "text": "Is this Chad GPT?", "start": 2023.893, "speaker": "SPEAKER_02"}, {"end": 2026.155, "text": "Who cares what it is?", "start": 2025.234, "speaker": "SPEAKER_02"}, {"end": 2029.718, "text": "If you feed it crap, it's going to respond with crap, right?", "start": 2026.695, "speaker": "SPEAKER_02"}, {"end": 2038.224, "text": " So you really need to take a lot of the document prep and the data pipelines and the cleanup and all this stuff super seriously.", "start": 2030.578, "speaker": "SPEAKER_02"}, {"end": 2042.648, "text": "We're dealing in a world now where people are like, well, what am I going to do as a designer?", "start": 2038.705, "speaker": "SPEAKER_02"}, {"end": 2043.688, "text": "You're going to do data cleanup.", "start": 2042.668, "speaker": "SPEAKER_02"}, {"end": 2051.174, "text": "You laugh, but it's like, you know, content strategy, anybody remember that?", "start": 2046.791, "speaker": "SPEAKER_02"}, {"end": 2054.117, "text": "That was like a hot button, I don't know, 20 years ago or something like that.", "start": 2051.214, "speaker": "SPEAKER_02"}, {"end": 2059.928, "text": " It's a big part of it, and this becomes more like direction, right?", "start": 2056.727, "speaker": "SPEAKER_02"}, {"end": 2064.789, "text": "Art direction, content direction, model direction, and steering.", "start": 2060.908, "speaker": "SPEAKER_02"}, {"end": 2069.891, "text": "Somebody needs to have an opinion when I was showing you those two answers on what is design.", "start": 2065.35, "speaker": "SPEAKER_02"}, {"end": 2074.392, "text": "Somebody needs to have an opinion on what is the answer that you want to deliver with the system.", "start": 2069.911, "speaker": "SPEAKER_02"}, {"end": 2075.092, "text": "What should it be?", "start": 2074.452, "speaker": "SPEAKER_02"}, {"end": 2076.793, "text": "What can't it be?", "start": 2075.933, "speaker": "SPEAKER_02"}, {"end": 2078.753, "text": "What is not okay?", "start": 2077.833, "speaker": "SPEAKER_02"}, {"end": 2080.614, "text": "What is okay?", "start": 2080.034, "speaker": "SPEAKER_02"}, {"end": 2095.652, "text": " Now, we could do even better, because recent advances, and we did this about a month or two ago, in vision models have made it possible for us to totally change the game and what we can do from images and PDFs.", "start": 2083.365, "speaker": "SPEAKER_02"}, {"end": 2100.334, "text": "So before, we spent a bunch of time trying to get models to do text extraction.", "start": 2096.372, "speaker": "SPEAKER_02"}, {"end": 2103.796, "text": "We got pretty good at extracting all the real content from a file.", "start": 2100.594, "speaker": "SPEAKER_02"}, {"end": 2111.2, "text": "But now what we can do is actually generate things that say something like, the chart illustrates the impact of A-B testing results from 2012 to 2019.", "start": 2104.316, "speaker": "SPEAKER_02"}, {"end": 2121.824, "text": " The gray line represents this, suggesting a discrepancy between the positive outcomes indicated by A-B testing and actual business results achieved.", "start": 2113.021, "speaker": "SPEAKER_02"}, {"end": 2125.465, "text": "What do you think is going to make your responses better?", "start": 2123.105, "speaker": "SPEAKER_02"}, {"end": 2127.326, "text": "This one.", "start": 2126.966, "speaker": "SPEAKER_02"}, {"end": 2129.547, "text": "Here's another example.", "start": 2128.687, "speaker": "SPEAKER_02"}, {"end": 2132.468, "text": "Here's a screenshot from my Mind the Gap talk.", "start": 2129.667, "speaker": "SPEAKER_02"}, {"end": 2134.028, "text": "This is what we had before.", "start": 2132.928, "speaker": "SPEAKER_02"}, {"end": 2138.07, "text": "Good job getting all that content and image out of there.", "start": 2135.229, "speaker": "SPEAKER_02"}, {"end": 2139.77, "text": "That's a pretty tough task.", "start": 2138.27, "speaker": "SPEAKER_02"}, {"end": 2140.731, "text": "Did a good job.", "start": 2140.131, "speaker": "SPEAKER_02"}, {"end": 2143.016, "text": " This is so much better.", "start": 2141.755, "speaker": "SPEAKER_02"}, {"end": 2154.205, "text": "And so now what we do is we have a pipeline that creates this kind of content from every single screen in the PDF and from every single image.", "start": 2146.519, "speaker": "SPEAKER_02"}, {"end": 2165.634, "text": "And when you then apply that to your embedding index and you change the content again, you once more have a very different set of answers for what is design.", "start": 2155.386, "speaker": "SPEAKER_02"}, {"end": 2167.996, "text": "And in this case, I think it's getting better.", "start": 2165.654, "speaker": "SPEAKER_02"}, {"end": 2170.678, "text": "Like every time I've done this, this answer has gotten better.", "start": 2168.016, "speaker": "SPEAKER_02"}, {"end": 2181.082, "text": " Every time we've enriched the content and spent the time to make sure that what we're feeding in and how we're processing it is a lot better.", "start": 2171.959, "speaker": "SPEAKER_02"}, {"end": 2183.703, "text": "So this is a process of constant adjustment.", "start": 2181.462, "speaker": "SPEAKER_02"}, {"end": 2187.084, "text": "It's the same thing like people say, OK, I launched the website.", "start": 2184.203, "speaker": "SPEAKER_02"}, {"end": 2187.464, "text": "I'm done.", "start": 2187.104, "speaker": "SPEAKER_02"}, {"end": 2188.164, "text": "No, you're not.", "start": 2187.664, "speaker": "SPEAKER_02"}, {"end": 2189.465, "text": "That's the beginning of it.", "start": 2188.504, "speaker": "SPEAKER_02"}, {"end": 2191.405, "text": "Oh, I added a chat bot.", "start": 2190.345, "speaker": "SPEAKER_02"}, {"end": 2191.786, "text": "I'm done.", "start": 2191.425, "speaker": "SPEAKER_02"}, {"end": 2193.046, "text": "No, that's the beginning of it.", "start": 2191.826, "speaker": "SPEAKER_02"}, {"end": 2208.633, "text": " So for PDFs, we need lots of cleaning, but now you can see that there's these, in addition to some of the text-based tasks, we've actually got some vision tasks to make sense of all that information and make it useful for these models.", "start": 2195.182, "speaker": "SPEAKER_02"}, {"end": 2220.303, "text": "The other really cool thing that this enables is once you start processing images and understanding semantically what's in an image, you can actually start to respond with images and get multimodal.", "start": 2209.354, "speaker": "SPEAKER_02"}, {"end": 2223.766, "text": " So now your website doesn't have to just like spit out text.", "start": 2221.044, "speaker": "SPEAKER_02"}, {"end": 2225.007, "text": "It can actually do things like this.", "start": 2223.906, "speaker": "SPEAKER_02"}, {"end": 2228.69, "text": "So I ask a question, what's the difference between AR and VR?", "start": 2225.508, "speaker": "SPEAKER_02"}, {"end": 2232.733, "text": "And like literally, this image shows the spectrum from VR to AR.", "start": 2229.191, "speaker": "SPEAKER_02"}, {"end": 2242.501, "text": "You can see that it creates a completely digital world while AR, it's not only putting the image in the right place in the response, but it's actually referencing it in its response.", "start": 2232.814, "speaker": "SPEAKER_02"}, {"end": 2246.745, "text": "So this creates a whole new set of capabilities.", "start": 2244.283, "speaker": "SPEAKER_02"}, {"end": 2253.542, "text": " Now you can start to respond with pages of PDFs, too, in the same way.", "start": 2248.52, "speaker": "SPEAKER_02"}, {"end": 2260.925, "text": "It knows what's in that page of the PDF because we've spent all this time to process it and put it in the embedding index.", "start": 2253.602, "speaker": "SPEAKER_02"}, {"end": 2268.568, "text": "We've trained a retriever to find it when it's most relevant, and then we can start to assemble answers that make a ton more sense.", "start": 2261.425, "speaker": "SPEAKER_02"}, {"end": 2271.669, "text": "Okay, that's a lot of stuff.", "start": 2270.388, "speaker": "SPEAKER_02"}, {"end": 2276.353, "text": " But hopefully, big picture, let's take a step back.", "start": 2273.612, "speaker": "SPEAKER_02"}, {"end": 2278.094, "text": "You have website.", "start": 2277.054, "speaker": "SPEAKER_02"}, {"end": 2280.475, "text": "Website have stuff in it.", "start": 2279.074, "speaker": "SPEAKER_02"}, {"end": 2281.636, "text": "Yes?", "start": 2281.395, "speaker": "SPEAKER_02"}, {"end": 2285.317, "text": "Okay, you all know website have stuff.", "start": 2282.876, "speaker": "SPEAKER_02"}, {"end": 2287.418, "text": "Different kind of stuff.", "start": 2286.338, "speaker": "SPEAKER_02"}, {"end": 2291.7, "text": "AI models on stuff.", "start": 2288.699, "speaker": "SPEAKER_02"}, {"end": 2294.501, "text": "Very cool.", "start": 2293.961, "speaker": "SPEAKER_02"}, {"end": 2298.063, "text": "Whole new ball game.", "start": 2297.163, "speaker": "SPEAKER_02"}, {"end": 2304.195, "text": " And I think we're just scratching at the surface of this.", "start": 2301.914, "speaker": "SPEAKER_02"}, {"end": 2312.2, "text": "By the way, this is why I have these little AI guys, because they're hustling through all these little AI models, trying to figure out what to do and how to use what thing, when, and how.", "start": 2304.235, "speaker": "SPEAKER_02"}, {"end": 2318.323, "text": "Behind the scenes, they're using those language models, those vision models, everything.", "start": 2314.281, "speaker": "SPEAKER_02"}, {"end": 2324.506, "text": "And so the end result here is not some... Hold on.", "start": 2319.123, "speaker": "SPEAKER_02"}, {"end": 2326.928, "text": "And the end result here is not some...", "start": 2325.387, "speaker": "SPEAKER_02"}, {"end": 2338.317, "text": " Scary AI thing, but instead it's a new way to use all this content, right?", "start": 2333.214, "speaker": "SPEAKER_02"}, {"end": 2342.219, "text": "To kind of mirror real world experiences.", "start": 2338.817, "speaker": "SPEAKER_02"}, {"end": 2352.484, "text": "You don't have to go through those 10 blue links and click on them and then find the part of the article that kind of relates to the thing that you're trying to go browse through long lists of things.", "start": 2342.239, "speaker": "SPEAKER_02"}, {"end": 2356.686, "text": "You can just say, what you're asking, right?", "start": 2352.504, "speaker": "SPEAKER_02"}, {"end": 2361.309, "text": "And so you could say something like, hey, what should designers do about AI?", "start": 2357.687, "speaker": "SPEAKER_02"}, {"end": 2365.12, "text": " Yeah, it's going to reference some talks.", "start": 2363.479, "speaker": "SPEAKER_02"}, {"end": 2367.082, "text": "It's going to give you an answer.", "start": 2365.24, "speaker": "SPEAKER_02"}, {"end": 2368.823, "text": "And away you go.", "start": 2368.283, "speaker": "SPEAKER_02"}, {"end": 2376.43, "text": "So these conversational interfaces, whatever we want to call them, or whatever they become in the future.", "start": 2370.625, "speaker": "SPEAKER_02"}, {"end": 2378.792, "text": "I launched this service on my website.", "start": 2377.11, "speaker": "SPEAKER_02"}, {"end": 2380.493, "text": "By the way, you can all, I should have said this again.", "start": 2378.812, "speaker": "SPEAKER_02"}, {"end": 2381.494, "text": "You can all go play with it.", "start": 2380.513, "speaker": "SPEAKER_02"}, {"end": 2383.616, "text": "Ask.LukeW.com.", "start": 2381.674, "speaker": "SPEAKER_02"}, {"end": 2385.377, "text": "Go ahead, ask whatever questions you want.", "start": 2383.656, "speaker": "SPEAKER_02"}, {"end": 2386.338, "text": "Don't crash it.", "start": 2385.697, "speaker": "SPEAKER_02"}, {"end": 2393.495, "text": " But we've been doing this for almost a year and a half now.", "start": 2389.352, "speaker": "SPEAKER_02"}, {"end": 2398.559, "text": "And like every couple months, there's some big interesting change where we can do new things.", "start": 2393.715, "speaker": "SPEAKER_02"}, {"end": 2400.62, "text": "Now we're doing these inline citations.", "start": 2398.819, "speaker": "SPEAKER_02"}, {"end": 2402.241, "text": "Now we're putting images in there.", "start": 2400.68, "speaker": "SPEAKER_02"}, {"end": 2405.023, "text": "And the models are actually referencing the images.", "start": 2402.702, "speaker": "SPEAKER_02"}, {"end": 2410.027, "text": "Next up, we're going to try doing that within inline video files.", "start": 2405.624, "speaker": "SPEAKER_02"}, {"end": 2411.088, "text": "Who knows?", "start": 2410.747, "speaker": "SPEAKER_02"}, {"end": 2411.988, "text": "Lots of stuff to do.", "start": 2411.248, "speaker": "SPEAKER_02"}, {"end": 2418.033, "text": "So on behalf of me and all my little AI guys,", "start": 2413.029, "speaker": "SPEAKER_02"}, {"end": 2421.723, "text": " Thank you.", "start": 2421.343, "speaker": "SPEAKER_02"}, {"end": 2433.629, "text": "And I have one question for you.", "start": 2432.369, "speaker": "SPEAKER_02"}, {"end": 2436.571, "text": "Are we going to actually find out what the bananas are for?", "start": 2434.17, "speaker": "SPEAKER_02"}, {"end": 2441.093, "text": "Because they've been sitting there all day, and I'm just dying to know.", "start": 2436.631, "speaker": "SPEAKER_02"}, {"end": 2442.694, "text": "OK.", "start": 2442.374, "speaker": "SPEAKER_02"}, {"end": 2444.175, "text": "No, no, no.", "start": 2443.775, "speaker": "SPEAKER_02"}, {"end": 2444.495, "text": "Thank you.", "start": 2444.195, "speaker": "SPEAKER_02"}, {"end": 2444.935, "text": "That's fine.", "start": 2444.515, "speaker": "SPEAKER_02"}, {"end": 2445.856, "text": "All right.", "start": 2445.516, "speaker": "SPEAKER_00"}, {"end": 2447.837, "text": "They're still here.", "start": 2447.157, "speaker": "SPEAKER_02"}, {"end": 2448.737, "text": "That's right.", "start": 2448.037, "speaker": "SPEAKER_00"}, {"end": 2449.658, "text": "Toby, you want a banana?", "start": 2448.877, "speaker": "SPEAKER_00"}, {"end": 2454.111, "text": " I mean, I'm not going to throw bananas at you.", "start": 2452.07, "speaker": "SPEAKER_00"}, {"end": 2456.132, "text": "That's a bit disrespectful, I think.", "start": 2454.151, "speaker": "SPEAKER_00"}, {"end": 2459.713, "text": "Thank you so much, Luke, for the insights.", "start": 2456.992, "speaker": "SPEAKER_00"}, {"end": 2461.254, "text": "Can we switch?", "start": 2460.674, "speaker": "SPEAKER_00"}, {"end": 2461.934, "text": "Oh, sure.", "start": 2461.614, "speaker": "SPEAKER_00"}, {"end": 2463.395, "text": "Am I in your chair?", "start": 2462.194, "speaker": "SPEAKER_02"}, {"end": 2466.976, "text": "No, I just feel... I have to sit near the bananas after that comment.", "start": 2463.455, "speaker": "SPEAKER_02"}, {"end": 2468.477, "text": "Oh, this is my spot.", "start": 2467.516, "speaker": "SPEAKER_00"}, {"end": 2474.852, "text": " Oh, they blend in, that's okay.", "start": 2473.311, "speaker": "SPEAKER_00"}, {"end": 2486.562, "text": "So I'm wondering at this point, actually when you think about like a selection of a model, I mean you basically said that it doesn't matter as long as you have like the proper information architecture, content strategy in place and all.", "start": 2475.173, "speaker": "SPEAKER_00"}, {"end": 2491.647, "text": "So you would say that there is basically no preference in terms of what you're going for?", "start": 2486.983, "speaker": "SPEAKER_00"}, {"end": 2509.865, "text": " It depends on what you're trying to do so if you want a general high-level Open-ended task a very large foundational model Claude GPT those things is going to be off the shelf what you want because it's post-trained if you want to do a very specific task", "start": 2492.808, "speaker": "SPEAKER_02"}, {"end": 2521.031, "text": " That benefit of being very broadly trained to do anything is actually a negative because they're not specifically trained to do one particular thing a very particular way.", "start": 2510.726, "speaker": "SPEAKER_02"}, {"end": 2521.411, "text": "Right.", "start": 2521.231, "speaker": "SPEAKER_00"}, {"end": 2530.596, "text": "So does it mean that then it's in some specific cases like an enterprise world where you want to kind of really optimize a specific task instead of going for large language models to go for small language models?", "start": 2521.431, "speaker": "SPEAKER_00"}, {"end": 2534.24, "text": " The architecture is usually a combination of multiple things.", "start": 2531.176, "speaker": "SPEAKER_02"}, {"end": 2539.627, "text": "So you use smaller fine-tuned models for very specific tasks where you want to do something concrete.", "start": 2534.941, "speaker": "SPEAKER_02"}, {"end": 2541.79, "text": "Let's talk about Brad's example earlier.", "start": 2539.687, "speaker": "SPEAKER_02"}, {"end": 2545.995, "text": "If I want my particular pattern library,", "start": 2542.291, "speaker": "SPEAKER_02"}, {"end": 2559.029, "text": " to be a very strong influence on the creation of components, then I'm going to create a bunch of training examples and I'm going to fine tune a model that's already good at writing code to write exactly my specific kind of code.", "start": 2546.616, "speaker": "SPEAKER_02"}, {"end": 2567.478, "text": "But in that same pipeline, I may use something bigger to answer like very high level things like make a plan for what this web page should be.", "start": 2559.429, "speaker": "SPEAKER_02"}, {"end": 2579.644, "text": " Right, so that is a very broad high-level world knowledge task and then like turn it into this code using this library is a very different kind of specific task I see use a smaller fine-tune model, right?", "start": 2568.298, "speaker": "SPEAKER_02"}, {"end": 2583.986, "text": "I'm also wondering just how come like actually those things like citations, right?", "start": 2579.884, "speaker": "SPEAKER_00"}, {"end": 2595.633, "text": "And and some of those nice these features that you find that in perplexity and in Claude like just sharing the actual output being able to Reef kind of fork it like what you get into this is fantastic, but most", "start": 2584.087, "speaker": "SPEAKER_00"}, {"end": 2598.034, "text": " AI experiences are not like that.", "start": 2596.613, "speaker": "SPEAKER_00"}, {"end": 2603.957, "text": "So I'm actually wondering, I mean, maybe that's my personal experience, but I just want to share and I'm wondering what you think as well.", "start": 2598.094, "speaker": "SPEAKER_00"}, {"end": 2609.24, "text": "I think over the last maybe three weeks, I have used Google once.", "start": 2604.717, "speaker": "SPEAKER_00"}, {"end": 2611.881, "text": "Maybe just once.", "start": 2611.101, "speaker": "SPEAKER_00"}, {"end": 2614.142, "text": "I mean, Google Maps is a different story, right?", "start": 2612.441, "speaker": "SPEAKER_00"}, {"end": 2617.484, "text": "But I mean, like, like literally searching and going through links, really rare.", "start": 2614.162, "speaker": "SPEAKER_00"}, {"end": 2622.427, "text": "But at the same time, over the last two months, I maybe used CHGPT once.", "start": 2617.944, "speaker": "SPEAKER_00"}, {"end": 2627.661, "text": " So the only thing I'm paying for is perplexity, and I'm using it all the time.", "start": 2623.84, "speaker": "SPEAKER_00"}, {"end": 2629.641, "text": "And I mean, like, all the time.", "start": 2627.961, "speaker": "SPEAKER_00"}, {"end": 2630.942, "text": "And I get the reference.", "start": 2629.681, "speaker": "SPEAKER_00"}, {"end": 2633.702, "text": "Perplexity employee.", "start": 2632.582, "speaker": "SPEAKER_00"}, {"end": 2634.782, "text": "Perplexity?", "start": 2633.982, "speaker": "SPEAKER_00"}, {"end": 2635.402, "text": "No, no.", "start": 2634.862, "speaker": "SPEAKER_00"}, {"end": 2637.143, "text": "Perplexity stock owner.", "start": 2635.923, "speaker": "SPEAKER_00"}, {"end": 2645.745, "text": "But I think for me, the reason why I'm saying that is that, to me, perplexity is doing this incredible job and kind of combining all these different patterns that you were also mentioning.", "start": 2637.263, "speaker": "SPEAKER_00"}, {"end": 2648.125, "text": "But why is it not more like a norm?", "start": 2646.425, "speaker": "SPEAKER_00"}, {"end": 2649.966, "text": "Why is the norm still just chatbot?", "start": 2648.145, "speaker": "SPEAKER_00"}, {"end": 2651.646, "text": "Why are most web pages pieces of shit?", "start": 2650.006, "speaker": "SPEAKER_00"}, {"end": 2662.552, "text": " Yeah, but, well, if you look at your GPT, right, it's, you know, over two years, it feels like it's better, but not better, if it makes sense.", "start": 2652.206, "speaker": "SPEAKER_00"}, {"end": 2666.814, "text": "My joke was not a joke, really, right?", "start": 2664.253, "speaker": "SPEAKER_02"}, {"end": 2675.178, "text": "Like, if you look at the early stages of mobile apps, the vast majority of mobile apps at the beginning were moving web pages into a tiny little screen.", "start": 2666.874, "speaker": "SPEAKER_02"}, {"end": 2675.839, "text": "They were pretty bad.", "start": 2675.218, "speaker": "SPEAKER_02"}, {"end": 2691.893, "text": " They're trying to apply very prior patterns to a new medium, and it just takes a while to learn how to use that medium, what the experiences should be, and there's going to be more evolution beyond what I showed and what perplexity is doing, whatever.", "start": 2676.739, "speaker": "SPEAKER_02"}, {"end": 2694.176, "text": "We're still super early in this stuff.", "start": 2692.334, "speaker": "SPEAKER_02"}, {"end": 2699.677, "text": " But at the same time, I feel like there is a bit, I mean, we had a very intense discussion about that yesterday.", "start": 2694.716, "speaker": "SPEAKER_00"}, {"end": 2703.918, "text": "Because again, when you look at how AI is perceived outside of the US, it's very different.", "start": 2700.157, "speaker": "SPEAKER_00"}, {"end": 2708.159, "text": "Like in Europe, whenever you start having conversations, there was a conference around you at AI.", "start": 2703.958, "speaker": "SPEAKER_00"}, {"end": 2714.321, "text": "And then there is this sense of frustration, I guess, which would be the right word.", "start": 2708.699, "speaker": "SPEAKER_00"}, {"end": 2720.562, "text": "And the reason for that is very simple, because many people are speaking about, you know, 2022 AI comes up, everybody's like, wow, that's magical.", "start": 2714.341, "speaker": "SPEAKER_00"}, {"end": 2732.168, "text": " Now, everybody's like, when people speak about AI, it's like it feels fake or hallucinating or slow because you always have to wait for your GPT out.", "start": 2721.542, "speaker": "SPEAKER_00"}, {"end": 2733.669, "text": "You can't render hands.", "start": 2732.188, "speaker": "SPEAKER_02"}, {"end": 2734.249, "text": "What's wrong?", "start": 2733.729, "speaker": "SPEAKER_02"}, {"end": 2735.37, "text": "Yeah, like all those things.", "start": 2734.309, "speaker": "SPEAKER_00"}, {"end": 2739.032, "text": "But I mean, I'm just wondering, where are we now with the idea?", "start": 2735.63, "speaker": "SPEAKER_00"}, {"end": 2741.553, "text": "Do we feel like it's like really groundbreaking innovation?", "start": 2739.052, "speaker": "SPEAKER_00"}, {"end": 2746.516, "text": "Because everybody was expecting, it felt like a big breakthrough in health care and whatnot.", "start": 2741.593, "speaker": "SPEAKER_00"}, {"end": 2749.218, "text": " Is it where we are?", "start": 2748.297, "speaker": "SPEAKER_00"}, {"end": 2750.118, "text": "Or is it not?", "start": 2749.278, "speaker": "SPEAKER_00"}, {"end": 2751.759, "text": "What is the state of it now?", "start": 2750.158, "speaker": "SPEAKER_00"}, {"end": 2755.1, "text": "I mean, I literally showed you a digital version of my brain.", "start": 2751.979, "speaker": "SPEAKER_02"}, {"end": 2759.402, "text": "Which is fantastic to see, but you also have it all documented neatly, I think.", "start": 2755.601, "speaker": "SPEAKER_00"}, {"end": 2764.145, "text": "We should have digital versions of your brain and everybody else's brains.", "start": 2760.223, "speaker": "SPEAKER_02"}, {"end": 2770.003, "text": " And then you have this... Then I'll know what you're doing with these.", "start": 2767.682, "speaker": "SPEAKER_00"}, {"end": 2771.744, "text": "Oh, yeah, yeah, yeah.", "start": 2770.784, "speaker": "SPEAKER_02"}, {"end": 2772.485, "text": "That's right.", "start": 2772.064, "speaker": "SPEAKER_02"}, {"end": 2772.865, "text": "That's right.", "start": 2772.505, "speaker": "SPEAKER_02"}, {"end": 2775.206, "text": "But do you see... No, it's the same answer.", "start": 2773.125, "speaker": "SPEAKER_02"}, {"end": 2776.307, "text": "We're still so early.", "start": 2775.266, "speaker": "SPEAKER_02"}, {"end": 2776.867, "text": "Yeah.", "start": 2776.687, "speaker": "SPEAKER_01"}, {"end": 2777.828, "text": "We're going to figure this out.", "start": 2776.947, "speaker": "SPEAKER_02"}, {"end": 2784.231, "text": "Like, when the iPhone came out in 2007, was there a TikTok?", "start": 2777.868, "speaker": "SPEAKER_02"}, {"end": 2786.653, "text": "Nope.", "start": 2786.452, "speaker": "SPEAKER_02"}, {"end": 2790.715, "text": "It's like everybody's brain now fused to TikTok in a terrible way.", "start": 2787.153, "speaker": "SPEAKER_02"}, {"end": 2790.895, "text": "Yeah.", "start": 2790.775, "speaker": "SPEAKER_02"}, {"end": 2793.019, "text": " Right.", "start": 2792.739, "speaker": "SPEAKER_00"}, {"end": 2807.655, "text": "So like those, it's just, you know, there's a ton of time which will put, so if you ask me, I think as I tried to illustrate there, you can basically put AI throughout your whole stack.", "start": 2793.079, "speaker": "SPEAKER_02"}, {"end": 2823.209, "text": " From the user experience all the way down to how you store information right to like what your data structures are or no data structures whatsoever your data structures are just text and then you can basically build up and down from there like that's.", "start": 2808.476, "speaker": "SPEAKER_02"}, {"end": 2824.75, "text": " Pretty crazy.", "start": 2823.869, "speaker": "SPEAKER_02"}, {"end": 2835.44, "text": "Even in all these other previous kind of like major technology changes we had, we didn't have anything that is the entire stack all at once.", "start": 2825.13, "speaker": "SPEAKER_02"}, {"end": 2840.004, "text": "Parts of the stack changed and that had massive repercussions on the industry.", "start": 2835.66, "speaker": "SPEAKER_02"}, {"end": 2844.889, "text": "Web, mobile, you name it, cloud, more on the lower side of the stack.", "start": 2840.224, "speaker": "SPEAKER_02"}, {"end": 2846.67, "text": "But now the whole stack", "start": 2845.289, "speaker": "SPEAKER_02"}, {"end": 2851.078, "text": " is basically being rethought and changed.", "start": 2847.771, "speaker": "SPEAKER_02"}, {"end": 2857.23, "text": "So I think the changes that we're gonna see of this, again, personally, I believe are gonna be more dramatic than those.", "start": 2851.539, "speaker": "SPEAKER_02"}, {"end": 2862.925, "text": " Actually, I should probably also cover some questions from the audience, by the way.", "start": 2859.584, "speaker": "SPEAKER_00"}, {"end": 2864.586, "text": "We can talk about your questions.", "start": 2863.406, "speaker": "SPEAKER_00"}, {"end": 2866.346, "text": "No, no, that's okay.", "start": 2865.046, "speaker": "SPEAKER_00"}, {"end": 2868.527, "text": "We can have a conversation anyway.", "start": 2867.407, "speaker": "SPEAKER_00"}, {"end": 2873.208, "text": "There was a question that came up quickly.", "start": 2870.308, "speaker": "SPEAKER_00"}, {"end": 2875.509, "text": "Can you talk about content strategy for AI?", "start": 2873.248, "speaker": "SPEAKER_00"}, {"end": 2876.449, "text": "What does it mean?", "start": 2875.649, "speaker": "SPEAKER_00"}, {"end": 2882.091, "text": "How do you maintain your application model for not just existing, but, ooh, that just jumped.", "start": 2876.809, "speaker": "SPEAKER_00"}, {"end": 2884.131, "text": "But new content, yeah.", "start": 2883.171, "speaker": "SPEAKER_00"}, {"end": 2885.932, "text": "New content and something on that.", "start": 2884.151, "speaker": "SPEAKER_00"}, {"end": 2887.273, "text": " Fascinating topic.", "start": 2886.292, "speaker": "SPEAKER_02"}, {"end": 2889.274, "text": "I don't think anybody really knows yet.", "start": 2887.833, "speaker": "SPEAKER_02"}, {"end": 2903.761, "text": "The one thing I didn't show with the Ask Luke system that I can, if people are interested, is when questions come in that I don't have answers to inside my content corpus, but I actually have an answer in my head.", "start": 2890.194, "speaker": "SPEAKER_02"}, {"end": 2908.423, "text": "I've gotten into this workflow where I built an admin system where I just press a button and talk.", "start": 2904.341, "speaker": "SPEAKER_02"}, {"end": 2918.002, "text": " It records all that, like I don't have to write a blog article or anything like that, it just slurps it all up and that becomes part of the content corpus and it grows.", "start": 2909.798, "speaker": "SPEAKER_02"}, {"end": 2931.349, "text": "And so behind the scenes over the past year and a half, I've generated something like 500 equivalent articles, which imagine publishing 500 blog articles in a year.", "start": 2918.883, "speaker": "SPEAKER_02"}, {"end": 2933.23, "text": "You'd be crazy, right?", "start": 2931.929, "speaker": "SPEAKER_02"}, {"end": 2933.85, "text": "It doesn't happen.", "start": 2933.25, "speaker": "SPEAKER_02"}, {"end": 2934.351, "text": "But now,", "start": 2933.97, "speaker": "SPEAKER_02"}, {"end": 2942.755, "text": " Question comes in, I know what people are asking about, and if I have an answer, I just hit record, talk, don't care about the format, don't care about the process.", "start": 2935.211, "speaker": "SPEAKER_02"}, {"end": 2952.059, "text": "And so that is a very different way of thinking about content strategy, because it's such a precious thing before, and it's so structured.", "start": 2943.335, "speaker": "SPEAKER_02"}, {"end": 2958.362, "text": "And now you can, again, basically use natural language as your data structure.", "start": 2953.099, "speaker": "SPEAKER_02"}, {"end": 2963.988, "text": " I've personally found it a tough bit to flip in my head.", "start": 2961.025, "speaker": "SPEAKER_02"}, {"end": 2970.737, "text": "For a long time, I was still like, well, I have to then format it into an article, and I have to remove the parts that are... No, you don't.", "start": 2964.87, "speaker": "SPEAKER_02"}, {"end": 2974.717, "text": " Drop it all in there and let the system figure it out.", "start": 2972.355, "speaker": "SPEAKER_00"}, {"end": 2979.161, "text": "And when you think about this type of data consumption, that's another question that came up and processing.", "start": 2975.098, "speaker": "SPEAKER_00"}, {"end": 2981.684, "text": "How does it impact document format structures?", "start": 2979.242, "speaker": "SPEAKER_00"}, {"end": 2984.226, "text": "Are we moving away from PDFs for good?", "start": 2982.304, "speaker": "SPEAKER_00"}, {"end": 2988.25, "text": "No, because people still want PDFs as output, which is funny.", "start": 2985.387, "speaker": "SPEAKER_02"}, {"end": 2992.334, "text": "One of the requests I have is like, can you turn that answer into a PDF?", "start": 2989.651, "speaker": "SPEAKER_02"}, {"end": 2997.527, "text": " Well, now we can turn it into audio podcast.", "start": 2995.566, "speaker": "SPEAKER_00"}, {"end": 2999.608, "text": "Yeah, turn it into a podcast.", "start": 2997.567, "speaker": "SPEAKER_02"}, {"end": 3000.869, "text": "It's funny.", "start": 3000.409, "speaker": "SPEAKER_02"}, {"end": 3011.835, "text": "I always, not always, but I have in the past made fun of Microsoft Copilot because Microsoft Copilot sits alongside Microsoft Office and it's always like, you're making a Word document.", "start": 3000.909, "speaker": "SPEAKER_02"}, {"end": 3013.436, "text": "Do you want to turn it into a PowerPoint?", "start": 3011.855, "speaker": "SPEAKER_02"}, {"end": 3015.457, "text": "And then it's like, you're making a PowerPoint.", "start": 3013.956, "speaker": "SPEAKER_02"}, {"end": 3017.058, "text": "Do you want to turn it into a Word document?", "start": 3015.477, "speaker": "SPEAKER_02"}, {"end": 3019.68, "text": " What am I doing?", "start": 3018.458, "speaker": "SPEAKER_00"}, {"end": 3022.222, "text": "Right?", "start": 3019.7, "speaker": "SPEAKER_00"}, {"end": 3025.326, "text": "Pat is wondering how do you test AI for hallucination?", "start": 3022.543, "speaker": "SPEAKER_00"}, {"end": 3029.75, "text": "We are running a financial application and we can't suggest inaccurate recommendations or insights.", "start": 3025.886, "speaker": "SPEAKER_00"}, {"end": 3030.951, "text": "Accuracy is very critical.", "start": 3029.87, "speaker": "SPEAKER_00"}, {"end": 3036.117, "text": "We haven't started working on AI yet because we're afraid of giving wrong suggestions to our users.", "start": 3031.372, "speaker": "SPEAKER_00"}, {"end": 3044.087, "text": " Yep, so the reason why I did this little Ask Luke project year and a half ago or whatever was because of that exact question.", "start": 3036.437, "speaker": "SPEAKER_02"}, {"end": 3048.393, "text": "It's very difficult to assess, is this right or wrong?", "start": 3044.608, "speaker": "SPEAKER_02"}, {"end": 3051.217, "text": "When I showed the answer there, where did Luke go to school?", "start": 3048.453, "speaker": "SPEAKER_02"}, {"end": 3052.499, "text": "I know the answer.", "start": 3051.818, "speaker": "SPEAKER_02"}, {"end": 3053.44, "text": "Do any of you?", "start": 3052.999, "speaker": "SPEAKER_02"}, {"end": 3054.521, "text": " No.", "start": 3054.361, "speaker": "SPEAKER_02"}, {"end": 3057.163, "text": "So how can you tell if the system is right or wrong?", "start": 3054.701, "speaker": "SPEAKER_02"}, {"end": 3065.629, "text": "And the reason we built that system on my corpus is because I know all that content, and I could assess and verify what is wrong or right.", "start": 3057.824, "speaker": "SPEAKER_02"}, {"end": 3070.953, "text": "So when you have that capability, then you can train the system to be like, that's the correct answer.", "start": 3066.33, "speaker": "SPEAKER_02"}, {"end": 3071.734, "text": "That's the wrong answer.", "start": 3070.973, "speaker": "SPEAKER_02"}, {"end": 3072.494, "text": "That's the correct answer.", "start": 3071.754, "speaker": "SPEAKER_02"}, {"end": 3073.255, "text": "That's the wrong answer.", "start": 3072.514, "speaker": "SPEAKER_02"}, {"end": 3075.677, "text": "I literally have an interface that", "start": 3073.675, "speaker": "SPEAKER_02"}, {"end": 3082.855, "text": " our AI researcher made for me, which is like, vote on the response, which one is correct.", "start": 3077.472, "speaker": "SPEAKER_02"}, {"end": 3088.399, "text": "And I have sat there for hours being like left, right, left, right, left, right.", "start": 3083.216, "speaker": "SPEAKER_02"}, {"end": 3091.2, "text": "But that was, you know, a long time ago.", "start": 3088.779, "speaker": "SPEAKER_02"}, {"end": 3093.622, "text": "Now I think the systems are a lot better.", "start": 3091.661, "speaker": "SPEAKER_02"}, {"end": 3098.904, "text": " At that, and it's obviously an issue that everybody's aware of in the space.", "start": 3094.322, "speaker": "SPEAKER_02"}, {"end": 3103.006, "text": "I think it's becoming less and less of a concern.", "start": 3100.725, "speaker": "SPEAKER_02"}, {"end": 3106.848, "text": "And especially if you do the type of thing that I showed there, right?", "start": 3103.686, "speaker": "SPEAKER_02"}, {"end": 3113.151, "text": "It's not foolproof, but the odds of things going off the rails are like this as opposed to...", "start": 3107.208, "speaker": "SPEAKER_02"}, {"end": 3117.713, "text": " like this and it'll get better and better and they'll be more fine tuning that you can do it.", "start": 3113.951, "speaker": "SPEAKER_02"}, {"end": 3126.197, "text": "And so how much time did it take from the very beginning when you just had all the documents to more or less decent results?", "start": 3118.053, "speaker": "SPEAKER_00"}, {"end": 3129.139, "text": "This is the thing about AI.", "start": 3126.257, "speaker": "SPEAKER_02"}, {"end": 3134.141, "text": "You can very quickly get to a compelling demo that's like 80% or 90% of what you want.", "start": 3129.599, "speaker": "SPEAKER_02"}, {"end": 3155.426, "text": " Right and we saw a bunch of that today I think in some of the earlier presentations like oh look I type this into Blank and out pops a thing That's kind of what I want the next 10% is like 90% of the work right and that's true in other Domains as well, but it's particularly true in AI to stitch together something.", "start": 3136.322, "speaker": "SPEAKER_02"}, {"end": 3157.047, "text": "That's working pretty good", "start": 3155.446, "speaker": "SPEAKER_02"}, {"end": 3159.824, "text": " Super fast, very easy.", "start": 3158.243, "speaker": "SPEAKER_02"}, {"end": 3165.426, "text": "To make something that's really robust and works for a long time, really long slog, lots of work.", "start": 3159.884, "speaker": "SPEAKER_02"}, {"end": 3170.509, "text": "Maybe final question, then of course we'll have more conversations at the party, I'm sure.", "start": 3167.367, "speaker": "SPEAKER_00"}, {"end": 3174.15, "text": "Ben, that was expected.", "start": 3171.889, "speaker": "SPEAKER_00"}, {"end": 3179.132, "text": "Are you worried about artificial general intelligence, AGI?", "start": 3174.55, "speaker": "SPEAKER_00"}, {"end": 3180.373, "text": "You are?", "start": 3180.093, "speaker": "SPEAKER_00"}, {"end": 3182.654, "text": "It's not my question.", "start": 3181.554, "speaker": "SPEAKER_00"}, {"end": 3184.475, "text": "You're answering first.", "start": 3182.714, "speaker": "SPEAKER_00"}, {"end": 3187.206, "text": " Well, I'm from Europe, we have to.", "start": 3186.046, "speaker": "SPEAKER_00"}, {"end": 3188.027, "text": "Yes, you are.", "start": 3187.487, "speaker": "SPEAKER_00"}, {"end": 3190.588, "text": "Don't worry, you've already regulated it.", "start": 3188.047, "speaker": "SPEAKER_00"}, {"end": 3191.148, "text": "It's fine.", "start": 3190.668, "speaker": "SPEAKER_00"}, {"end": 3193.068, "text": "Yeah, we have to protect the humanity and all.", "start": 3191.168, "speaker": "SPEAKER_00"}, {"end": 3196.709, "text": "The law has been signed, so it's no longer an issue.", "start": 3193.789, "speaker": "SPEAKER_02"}, {"end": 3197.17, "text": "You're welcome.", "start": 3196.749, "speaker": "SPEAKER_00"}, {"end": 3199.01, "text": "So what are you thinking?", "start": 3197.97, "speaker": "SPEAKER_00"}, {"end": 3199.59, "text": "No.", "start": 3199.47, "speaker": "SPEAKER_00"}, {"end": 3201.031, "text": "What do you mean no?", "start": 3200.311, "speaker": "SPEAKER_00"}, {"end": 3202.831, "text": "I mean, like, are you worried?", "start": 3201.851, "speaker": "SPEAKER_00"}, {"end": 3204.372, "text": "Oh, I can't wait.", "start": 3203.091, "speaker": "SPEAKER_00"}, {"end": 3208.633, "text": "That's a different way of looking at it.", "start": 3206.432, "speaker": "SPEAKER_00"}, {"end": 3214.595, "text": "If there's machines that can do all the crap work that I have to do all day and I can just mountain bike all day, like bring it on.", "start": 3209.033, "speaker": "SPEAKER_02"}, {"end": 3218.861, "text": " All right, well, let's stay hopeful then, I would say.", "start": 3216.112, "speaker": "SPEAKER_00"}, {"end": 3221.068, "text": "All right, thank you so much, Luc, for being with us.", "start": 3219.303, "speaker": "SPEAKER_00"}], "transcriptBySpeakers": [{"end": 10.384, "text": "the last session of the day. And we have Luke with us. Look, lurking in the back. Hello, Luke. Hello. So we both are green.", "start": 0.009, "speaker": "SPEAKER_00"}, {"end": 12.696, "text": "Yes, you finally caught up.", "start": 11.536, "speaker": "SPEAKER_02"}, {"end": 86.742, "text": "We finally like this is like you match the slides. So did we? Yeah, the color partially. Yeah, there is a bit of that. So AI, huh? Still a thing. I was I think it's a thing. Somebody mentioned it earlier today. Yeah. Yeah. So we've been having conversation about, uh, well, many things yesterday. And then of course the AI came along and, oh my, look is all in, like all in 100% in AI is this. So what do you do? Anybody actually working on a product that now embeds or integrates AI in some way? A few people? Okay. Who has given up on AI? Like AI is like so 2022. Okay. A few people, a few people, maybe, maybe look will change your mind today as well. Apparently, AI has eaten, well, looks website. Don't give it away. I should. Okay. I will not. So, but let's see. I mean, again, looking at what we, not, not what we can do with the AI, but maybe what we should do with the AI and how it all works under the hood. And also just for you to know, of course, let me just drop, drop in back in here. So Lucas Karanti, Managing Director, Sato Whittle Hill Ventures, but also earlier was a product director at Google and spent a lot of time of his life with little things like web form design. Remember that? Yeah, I do. 2010. Irrelevant now.", "start": 12.716, "speaker": "SPEAKER_00"}, {"end": 91.682, "text": "And mobile? Yeah, mobile, also irrelevant.", "start": 89.241, "speaker": "SPEAKER_02"}, {"end": 100.024, "text": "Well, then, I don't know what to expect now. Well, AI, I guess. So, please, round of applause to Lukro Blewski.", "start": 93.082, "speaker": "SPEAKER_00"}, {"end": 2444.935, "text": "Okay, thank you for having me. And, uh, there we go. And all of my little AI guys. Now, the staff here is so on top of it because we blew up these balloons and the moment you guys walked out the door, they took them all away. And I had an elaborate scheme for all of us stomping on the balloon at the same time and making a big noise. And now that has completely fallen apart. Thank you. But we're going to try this. Can you stand up for a second? You've been sitting really long. Your butt's probably hurt. Your brains are fried. Okay, so instead of stomping on a balloon, just try to make a big stomp noise. Okay, we have to time it, all right? Ready? So I'm here to talk about how AI ate my website, right? And when we think about that title, you might have some scary thoughts, AI, eating a website. What could that be? That doesn't sound very friendly, right? You might imagine some scary things. Okay, so on the count of three, I'm gonna try to time this. I want you to stomp a really scary stomp, and I'm gonna bring in a very scary AI thing. Okay, you ready? One, two, three. Oh, you did it! You conjured it! So here's a giant computer brain eating up all these web pages. Ah! My little guys! Let it go! Anybody scared? Okay, you can sit down. Don't worry, all the little A guys, AI guys, we'll be back. Woo! Okay. So in truth, when you think about AI, by the way, it's not as emphatic as when everybody pops a balloon. But that was good. I like that. Thank you. In truth, though, when most people think about AI, they think about something like chat GPT. We saw a bunch of examples of people using prompt engineering through chat GPT. This is a large language model. You can ask it questions like, who am I? Who is Luke? So I don't really need an intro slide anymore, because all this stuff is just available. But it's not just my website that these giant AI models have sort of subsumed and eaten up, right? They've also eaten up the entire rest of the web. And so when you ask a question like, where did Luke go to school? It's going to respond to something that seems really likely, right? Because people who do product designs very likely that they actually went to CMU. Probably many of them did. But I did not. So I say, I think Luke went to UIC and it responds, you're right, I'm so sorry, but he went to grad school at CMU, which I did not. So then I can say, did I really go to CMU? You're like, of course not, you're right. So no problem with reinforcement learning in these models. They're very oriented to doing what you say, right? You're wrong. I apologize. You're wrong again. I apologize. So we can kind of get them into whatever state we want. And sometimes they'll veer off into weird places, which I'm not going to do because the intro to this talk was weird enough. But let's zoom out for a second and look at this relationship. So we have websites, like my little website, where probably that information about me existed or maybe existed on the SmashingConf website. And you've got lots and lots of these websites, right? And all these things get basically turned into training data for these immense models. And they then can generate a response based on all the things they've learned about. What's the most likely thing to come next? And then they get tuned in different directions with things like reinforcement learning. So hence why it assumes that I as a product designer went to CMU. But this whole concept of training giant machine brains on the totality of published content on the internet You know, that doesn't sound nice, right? It sounds kind of scary, hence my scary AI thing here, conjures these scary narratives. But it doesn't have to be this way, because if we stop thinking in terms of this monolithic giant AI brain, And we instead start thinking about all the things different AI models, large or small, can do. And then we start thinking about how they can actually help us create new user experiences, build products in different ways. We heard a little bit about those in the morning sessions. There's a lot of things these things can do with language. They can basically take speech, turn into text. They can take text, turn it into speech. They can transform languages from one to the other. With vision, they can identify objects, identify faces. They can segment stuff. There's this huge list of capabilities that we now have because of these very capable AI models. And what's different from a few years ago, we have always had, not always, but machine learning's been around for a long time. But the dramatic shift over the past maybe two-ish years is that these very, very capable models are now accessible to anyone, anywhere and everywhere. You can download them, you can run them from the cloud. And so all these capabilities are just flying all over the place and showing up in so many different places. So people are learning about all these capabilities and integrating them into lots of different places. And this is the way I really like to think about this change, as opposed to the scary AI brain. To me, it's much more like, what are the new ingredients that we currently have that we can use to rethink experiences, to do things we couldn't do before, right? To go from all of these different technologies to new capabilities, especially on the web. Because God bless Brad and his enthusiasm for the original principles of the web. The web is not a very cool place right now. You go there and you're just inudated with like 15 dialogues and subscriptions for newsletters and you can't even read the content and honestly the first thing you want to do is kind of leave. It's not very cool. But I think we can make it really cool. So with that in mind, I'm just gonna walk you through a whole bunch of details on putting these things to work. And the place where I've been putting them to work is the place that I can control the most, which is my little part of the worldwide web here. This is my website. I started this 28 years ago, so how many of you are less than 28 years old? Okay, yeah, that's what I thought. What can you do in 28 years of having a website so you have things to aspire to? Well, I've written 2,000 text articles, put up about 100-something videos, bunch of PDF presentations, made almost 2,000 images. Yes, you too can achieve such things. That's a lot of stuff. And the way you can make use of that stuff using a web browser or the internet today is mostly like two modalities. You can search for things, right? You come back with some results. You can browse around. But even just my little website has 500,000 written words, 6,000 minutes of video. So like, God bless you if you ever make your way even through part of that. Highly unlikely. But what if we didn't have to just do search and browse? What if we could actually mirror the real world? And the real world that we're in, Vitaly keeps reminding us about the Pac-Man, right? Talk to each other. What if you could just talk? And so you could ask me a question like, hey, Luke, how do I decide what metrics I should use for my app? And in the real world, what I would do is be like, hmm, what do I know about that? I'll think about maybe a bunch of the things I've written, maybe some of the talks I've given in the past. And then I, in my head, assemble an answer and really quickly respond to you. The human brain is an amazing thing, much more amazing than any of these AI models we have. But I would think about what I know and within, you know, fractions of a second be able to assemble it into a coherent response and just keep talking and answer you. That's pretty different than our options online. If you do search, how do I pick metrics from my app? Go fish through these links. If you're browsing around and you make your way to all the articles tagged with metrics, well, there's 243 of them. So again, good luck. But what if instead we can do what I'm saying? How do I pick metrics from my app? So go through everything I've written or said on the topic, put them together into a coherent response, cite the sources, so you can go even deeper or verify what I said. So in this case, this response, how do I pick metrics from my app? You could see if you put this in the little system, I'm going to show you. You get this answer. And one of it is coming from a talk called Mind the Gap from 1 Minute 17. one hour 17 minutes and 50 seconds you click on that and it will take you to that part of the video right there queued up and you can see on the slide in the background how do I decide what to measure so those citations are tied directly to where the information is coming from so This is what happens when a bunch of AI models effectively eat all the pieces of content that make up a website and spit up a very different interaction model. And the first question people always have is, well, how do you do that? Well, I'll get to that. But first, I want to look at what the experience is, and then I'll talk through the pieces of how it gets put together. Now, when you see this, I ask a question, you respond with an answer. Many people say, well, isn't this just a chatbot? What LLM did you use for this chatbot? Is this chat GPT? This is what most people start thinking. Why does chat keep showing up? Number one, chat is a paradigm that's extremely familiar to all of us. You all have probably spent way too much time in your messaging app, even today, some of you may be on it right now. So this whole paradigm of messaging people, very familiar, very ubiquitous in our lives. So it's quite easy to kind of get in and running with that. But the pattern of this is a series of alternating messages, right? I say something, someone replies, the list goes on, If a message runs long, the space for it, you know, you get what these things look like, right? But is that really a great interaction design pattern for communicating to a global super brain? Maybe. I don't know. Probably not, though. If you actually look and see some of the issues that come up with this type of pattern, and this is some research from Nielsen Norman, You know, it's not really optimal for an iterative conversation with a set of documents. So they did a bunch of tests on these large language model-based chat experiences. And in particular, people having to kind of scroll through these conversation threads to find things, remember what was what, kind of veer things in the right direction. And a quote on the bottom here I think is pretty meaningful, which is, this behavior is a significant point of friction for everybody. that they observed in these studies. So we could probably do a bit better. Now, to account for some of the things that they found in those studies, we made use of a different interaction pattern when creating the experience you saw a second ago. So instead of this back and forth chat, everything expands, what you'll find in this design is something that looks a little bit more like this. And so what does it actually look like? Basically, all the previous question and answer pairs get collapsed after you've messaged them or completed them. Then the current one that you're on will open up, show you lots of space for the answer. And you can kind of go back and forth within them. So let me just show you what this looks like rather than talking about diagrams. So here you can see, how does free shipping help? It's part of this conversation. There's the response. If I want to go back and see how do you provide real time feedback, then I open up that one. And how does free shipping help thing close up? So I can kind of access one question and answer pair at a time. And then I can focus on that specific question that I have and its response when I'm asking it. And the rest kind of just rolls up and gets out of the way. So you can much more easily scan, find the thing you were looking for before, and get back to it. And this interaction actually works a little bit more like a FAQ style pattern. You guys have probably seen these. So it's like a big ask thing, right? And then there's all these frequently asked questions that you can open up and expand. Pretty much most websites have this stuff. And so this is a nice example of when you start from the problem space, right? When you don't start with, oh, we need a chatbot. Let's implement a chatbot. And you think about, well, what are we trying to do? Well, people are asking questions. They're trying to get responses. It's tied to a specific set of information. Maybe there's a different interaction paradigm that makes a little bit more sense here. And this Q&A interface that I just showed you looks a little bit more like a FAQ pattern than a chatbot pattern. And I bring this up because too often the process of design starts with something like a competitive audit. Or let's go see what everybody else is doing, put it up on a wall, right? And then as soon as you do that, if you've ever ridden a bike and you're riding a bike and you look at a tree, where are you gonna go? You're gonna ride right into the tree, right? It's called target fixation. So as soon as you see that chatbot pattern on chatGPT, guess where your design is gonna go? Right into that pattern. And whether intentionally or not, you end up copying what these different things are doing instead of letting the problem space actually guide the solution. So in this case, starting with understanding that problem of people are asking questions, they need answers, there's going to be a lot of them, they're going to go back and try and reference things, they need a way to scan through stuff, we ended up a little bit more of a FAQ thing than a chatbot thing. So now there's this expandable conversational interface. You ask a question, response comes down, some citations come up. But for the purpose of this presentation, because these are big slides on a screen, I've simplified a few things here. So I actually want to show you two more things on how this interface works. And one important one is this concept of suggested content. So you could see here. there's a response and then immediately after the response there's a series of follow-on questions that are related to the question you just asked. The first couple are directly tied to some of the questions that show up there and then the other ones are broader and kind of expand to other things that you can ask. And these suggested questions are actually really critical in anything that is just this big open-ended Box. Because one of the big problems people have with these things is, what do I ask? I don't know. Well, you can ask anything. That's like saying, don't ask anything. Nobody knows. And so having something like this addresses the issue of what should I ask. And lots of people have that problem because something like 65 or 70% of all the questions that get asked in this interface come from these suggested questions, right? So they're very helpful for getting people started and moving them in the right direction. So we knew from the start these were important, but it took a bit to actually iterate through the designs to get to what you saw there. So at first we experimented with something like very explicit to trigger suggested questions, because we're like, oh, we don't want to show these all the time. Somebody should be able to kind of click on something and get those. So we started with this little light bulb, right? Give me an idea, get it? So you click the light bulb, bunch of suggested questions come up. But then you get the classic, nobody knows what that icon is, are they gonna click it? So then you try something like, what else can I ask? And that shows up. But in both cases, the burden is on people to ask for these things. So instead, here's a series of designs that actually just started dropping it right after every answer, you'd get these suggestions. And there was a couple iterations on this until finally, We ended up on the most obvious way you could possibly do this, which is the way design always works. It's obvious in retrospect. Just put it directly in line right after the answer. Make it look almost like the answers, except a tiny bit different and away you go. So these follow directly in line in the interface. And the other important detail that I'm not showing, which is something we just launched a week ago, maybe two weeks ago, is now we created this new capability and you have to like squint the CDC is little like ones and twos popping up over there. These now are direct citations of the part of the answer that relates to where that content came from. So you can kind of see within the response which information piece is exciting. And if you click on them, as I'll show you in a second, you can go and see that information. So further kind of making obvious the way this thing works. And the way this works is it's pulling all this information from my website, right? All those articles, all those videos, all those audio files, and using that information to create a bunch of responses. for people's questions about design. Now, if you see one of those little citations, you can click on them and it'll expand it directly in line. So to go back to my diagram before here, how does free shipping help? You can see it's saying this article on evolving e-commerce shipping was used to make this answer. You can open up and there's the article and you have the ability to ask questions just of that article or go back to this more global mode. So what kinds of things can answer questions is not just those articles. As I mentioned before, there's videos, podcasts, PDFs, images, and more. But how do you get this stuff to be usable in this kind of interface? You have web pages, they have PDFs, you have websites, they've got images, you've got all this stuff. How do we take those things and then use them in this type of interface? So there's a couple things that we need to do and this is where the ability for you to use a whole bunch of different AI models starts to, I think, emerge. So, for example, if we look at videos, we have like a video, like this talk, let's say, is recorded. There'll be a video file at the end of it. We can use a speech-to-text model to take that video file, turn it into text. We can use a diarization model to break up who's talking when, when there's the Q&A session. We can have different voices and split that up. We can generate summaries, keywords, and then we can use vision models to extract items in those video screenshots, people, that kind of thing. So we take these files, we run them through a bunch of AI tasks using a number of models, and then we have to do a bunch of cleanup of that content. And when I say cleanup, I mean like we make sure that it understands that Luke is Luke, right? So it kind of cleans up the entities. We add some metadata, like date, location, et cetera. And so now we have content that we can use to start answering people's questions that before was just a bunch of video files. And to show you how far you can go with this, I'll give you one example. One of the things that we wanted to do was actually get some nice looking thumbnails for these videos. As you can see here, they're kind of all over the place. And so what you can do pretty quickly is generate a bunch of thumbnails from a video. This is a pretty trivial script with something like FFMpeg or what have you. spit out a bunch of videos, and then I can manually just go through four or five of those videos, or up to ten or something like that, and pick a good looking thumbnail that I like. And then using that little set of data, here's a bunch of thumbnails, pick this one, here's a bunch of thumbnails, pick this one, as examples, we can fine tune a little AI model that for all the rest of the hundred something videos picks a thumbnail for us. So the next time it goes through here, it's going to use this lightweight training that we give to pick this beautiful image of me here with the Which Paris Girl Are You screenshot. It nailed that part. It left me with a dumb little grin, but whatever. It's probably the best thumbnail of the bunch. So this is just a quick example of how you can think about having AI models do a lot of things for you. If you can kind of model a behavior to use that word intentionally, and if you can show a machine how to do something, you can train that machine to start doing it. And that, I think, is a really big switch to go off in your head. All right, if I can show a machine how to do this, I can get the machine to do it pretty consistently, even little things like this. And so this opens up a lot of possibilities in terms of, again, how you can invert experiences because you can have machines do all this processing. For audio, we can do the same thing. Take a bunch of podcasts, take a bunch of audio interviews, turn them into text, diarize them, summarize them. Whatever you need, do a bit of processing to them. Again, same kind of thing for articles. Summarize them, extract keywords, generate suggested questions, all this good stuff. So we start with all of these files that make up your typical web page is, websites, audio, video, text. images, what have you. We use these different AI models to turn them into useful, common kinds of content. And then when we have everything in a common format, this is where some of the secret sauce happens. What we do is we take all those little bits of content, call them chunks, whatever you want to call them, and we basically put them into an embedding space. that puts related concepts to it. This isn't exactly how it works, but this is a good mental model to have. So, e-commerce-y things go over here. Mobile touchy things are over here. What have you? And in some cases, those are video pieces. In other cases, those are images. In other cases, those are text snippets. So all this stuff is in basically one big, you can think of it as a fuzzy search index or another way to think about it as a semantic search index where you're actually looking things up by their relative meaning as opposed to things like keywords or metadata look up. So I'll show a system diagram super quick just so folks get it like this always happens. But you start with your docs, you do a whole bunch of this cleaning, you stick them into this search index that organizes them again in a very fuzzy kind of semantic way. And then you need some way of searching against it. You can have multiple ways of searching against it. Back comes some results. You can rank those results. And then you take what you found and you basically use them to generate a response. You can use lots of different models if you want. You can use one model. You can come up with a number of answers, pick the best one, using another AI model if you get the idea. And then you get to a point where you've got the right information to answer people's questions. So we break all the things into little chunks. We stick them into this crazy vector space. And then when people ask questions, we go find the relevant pieces and assemble them into answers. It's AI models up and down. In fact, it goes even deeper. And hopefully, this is one thing that you take away from this. If you go down the rabbit hole, there's like an AI model everywhere in this kind of workflow. Because one of the important things to do is actually learn, right? You get somebody an answer, you want to make sure that your system gets smarter. So one of the things that we did a couple weeks ago is we took all of these lists of questions people ask, and then the list of corresponding documents that we used to answer them, and we trained a custom retriever. which then spits out a score for how good is this document in answering the specific question. And we did that with using that feedback that we were getting. And if you may say, well, why does that matter? If you just take some off-the-shelf AI model that sticks things into this embedding space, that kind of puts things in the vector space that you can later find it, those are just trained on general sets of sentences, like global internet stuff. So they'll be okay. at finding the pieces that are relevant in my content. But they're not going to be great. And if you make a custom one, these scores mean nothing to you except bigger is better. So that cross-encoder compared to just using an open AI model does a heck of a lot better. It's much, much better. So you can make your own little AI models that find relevant bits of content, that understand what kind of content you have, and what good content is for the specific kinds of questions that get asked on your sites. And the whole thing becomes a virtuous loop. So some of you may be thinking, oh my god, that's a lot of work. Why would we do this? Why wouldn't I just show GPT on my website? Well, if you ask ChadGPT a question like, why do so many designs look the same? You're going to get this generic world knowledge answer, right? It's going to be like, well, it's because of cognitive biases, legal and regulatory constraints, economic factors, the influence of trends and cultural influences. It just sounds like a lot of horse shit to me. So all that work I showed you actually allows us to deliver something that mirrors the way I would actually answer a question. Kind of like reflects my specific opinions of things. So instead, when you ask our interface, why do you so many designs look the same? I basically say, look, it's because designers are just looking what everybody else is doing and kind of copying it without thinking too much. very different kind of answer. And then the other thing you get is all these citations that I was highlighting earlier, where again, you've got a bunch of information you've published in whatever websites you have or whatever applications you have, you can reference that stuff, make sure that it's cited and allow people to go deeper into all of it. Now, when I say go deeper, it's worth showing a little bit of example of what that actually means. So we asked this question, why do so many designs look the same? And that answer comes from, I mentioned a few times, videos, audio, web pages, whatever. If you were to click on one of those videos, what we're going to actually do is inside this conversational interface, bring up that video. I showed you an example earlier. Take you to that part of the video where that answer is showing up inside of the video track, and then give you the ability to ask questions of it, give you a. Summary of that video give you a full transcript that you can search against, so you can click that transcript thing and look for whatever you want inside that video file. If it's an audio file, same thing, we'll just embed an audio player, give you again that transcript and summary, allow you to ask questions of these multi-hour long audio files right then and there. If it is an article, we'll put the article in there. If it's a PDF, same thing, you kind of get it. And so again, this is an example of how this interface, this conversational interface, has basically eaten all the stuff that previously made a website. Because now all these things appear in the conversation. You ask a question. Here's the video that answers it. Now you're watching that video. You ask that a question. Here's an audio file. Now you can be inside that audio file or inside that article or through a PDF. Now I said the word PDF and my body sort of cringed a little bit, because PDFs are special, as in very painfully special. When I was building this interface, we first started with blog articles, and then we added video, and then we added audio, and then we added PDF. And when we added something like 400-ish PDFs to here, you could see something changed pretty dramatically. So before the question you could ask, what is design? It would say something like, design is the art of gradually applying constraints. So I kind of like this answer over here. And then I added the 400 PDFs, and all of a sudden, This one PDF file from 2006 about dogma free design like just dominated the results. And now it says it is directed towards human beings with the aim of solving design is the fundamental soul of human creation. I'm like, what is this? This doesn't sound like me. Oh, what turned out is inside this PDF, there was like four screens of quotes from other designers about what is design. And so the model, the information retrieval model is like, hell yeah, look at this baby, right? And it's like. And so the answer just becomes like this regurgitation of a whole bunch of quotes about what is design. So that's not good, right? And the other thing that was happening was a lot of junk is kind of showing up as part of this content. So if you break down what's inside, a PDF is like markup language, right? It's got so much stuff in there about how things should look as opposed to what the content is. And so you just get pieces of content that are just crap. I mean, look at some of these things. 1,268, the content is zero nature. One, two, six, five, biography photo, quote, auto video in the press. So you're like, dear AI model, please answer the question with this. And surprising the thing works at all. It's actually a testament to how amazing these things that it works at all. So what we ended up doing is cleaning this up again, getting rid of a bunch of stuff that's crap and clogging up our answers, introducing a mechanism where we diversify the answer. So like, yo, yo, yo, you can't just use this one PDF even though it loves to have design quotes. Break it up a little bit. And when we did that, we actually got to this really nice balance answer for what is design. There's a PDF there, there's a video, there's two podcasts, there's a text file. And now it's starting to really represent the diversity of content and clean it up. So the answer got a heck of a lot better through cleaning up all this content and balancing what shows up when. And hopefully I'm getting through to you. You know, people ask, like, what chatbot did you use for this? What LLM is this? Is this Chad GPT? Who cares what it is? If you feed it crap, it's going to respond with crap, right? So you really need to take a lot of the document prep and the data pipelines and the cleanup and all this stuff super seriously. We're dealing in a world now where people are like, well, what am I going to do as a designer? You're going to do data cleanup. You laugh, but it's like, you know, content strategy, anybody remember that? That was like a hot button, I don't know, 20 years ago or something like that. It's a big part of it, and this becomes more like direction, right? Art direction, content direction, model direction, and steering. Somebody needs to have an opinion when I was showing you those two answers on what is design. Somebody needs to have an opinion on what is the answer that you want to deliver with the system. What should it be? What can't it be? What is not okay? What is okay? Now, we could do even better, because recent advances, and we did this about a month or two ago, in vision models have made it possible for us to totally change the game and what we can do from images and PDFs. So before, we spent a bunch of time trying to get models to do text extraction. We got pretty good at extracting all the real content from a file. But now what we can do is actually generate things that say something like, the chart illustrates the impact of A-B testing results from 2012 to 2019. The gray line represents this, suggesting a discrepancy between the positive outcomes indicated by A-B testing and actual business results achieved. What do you think is going to make your responses better? This one. Here's another example. Here's a screenshot from my Mind the Gap talk. This is what we had before. Good job getting all that content and image out of there. That's a pretty tough task. Did a good job. This is so much better. And so now what we do is we have a pipeline that creates this kind of content from every single screen in the PDF and from every single image. And when you then apply that to your embedding index and you change the content again, you once more have a very different set of answers for what is design. And in this case, I think it's getting better. Like every time I've done this, this answer has gotten better. Every time we've enriched the content and spent the time to make sure that what we're feeding in and how we're processing it is a lot better. So this is a process of constant adjustment. It's the same thing like people say, OK, I launched the website. I'm done. No, you're not. That's the beginning of it. Oh, I added a chat bot. I'm done. No, that's the beginning of it. So for PDFs, we need lots of cleaning, but now you can see that there's these, in addition to some of the text-based tasks, we've actually got some vision tasks to make sense of all that information and make it useful for these models. The other really cool thing that this enables is once you start processing images and understanding semantically what's in an image, you can actually start to respond with images and get multimodal. So now your website doesn't have to just like spit out text. It can actually do things like this. So I ask a question, what's the difference between AR and VR? And like literally, this image shows the spectrum from VR to AR. You can see that it creates a completely digital world while AR, it's not only putting the image in the right place in the response, but it's actually referencing it in its response. So this creates a whole new set of capabilities. Now you can start to respond with pages of PDFs, too, in the same way. It knows what's in that page of the PDF because we've spent all this time to process it and put it in the embedding index. We've trained a retriever to find it when it's most relevant, and then we can start to assemble answers that make a ton more sense. Okay, that's a lot of stuff. But hopefully, big picture, let's take a step back. You have website. Website have stuff in it. Yes? Okay, you all know website have stuff. Different kind of stuff. AI models on stuff. Very cool. Whole new ball game. And I think we're just scratching at the surface of this. By the way, this is why I have these little AI guys, because they're hustling through all these little AI models, trying to figure out what to do and how to use what thing, when, and how. Behind the scenes, they're using those language models, those vision models, everything. And so the end result here is not some... Hold on. And the end result here is not some... Scary AI thing, but instead it's a new way to use all this content, right? To kind of mirror real world experiences. You don't have to go through those 10 blue links and click on them and then find the part of the article that kind of relates to the thing that you're trying to go browse through long lists of things. You can just say, what you're asking, right? And so you could say something like, hey, what should designers do about AI? Yeah, it's going to reference some talks. It's going to give you an answer. And away you go. So these conversational interfaces, whatever we want to call them, or whatever they become in the future. I launched this service on my website. By the way, you can all, I should have said this again. You can all go play with it. Ask.LukeW.com. Go ahead, ask whatever questions you want. Don't crash it. But we've been doing this for almost a year and a half now. And like every couple months, there's some big interesting change where we can do new things. Now we're doing these inline citations. Now we're putting images in there. And the models are actually referencing the images. Next up, we're going to try doing that within inline video files. Who knows? Lots of stuff to do. So on behalf of me and all my little AI guys, Thank you. And I have one question for you. Are we going to actually find out what the bananas are for? Because they've been sitting there all day, and I'm just dying to know. OK. No, no, no. Thank you. That's fine.", "start": 100.044, "speaker": "SPEAKER_02"}, {"end": 2445.856, "text": "All right.", "start": 2445.516, "speaker": "SPEAKER_00"}, {"end": 2447.837, "text": "They're still here.", "start": 2447.157, "speaker": "SPEAKER_02"}, {"end": 2461.934, "text": "That's right. Toby, you want a banana? I mean, I'm not going to throw bananas at you. That's a bit disrespectful, I think. Thank you so much, Luke, for the insights. Can we switch? Oh, sure.", "start": 2448.037, "speaker": "SPEAKER_00"}, {"end": 2466.976, "text": "Am I in your chair? No, I just feel... I have to sit near the bananas after that comment.", "start": 2462.194, "speaker": "SPEAKER_02"}, {"end": 2491.647, "text": "Oh, this is my spot. Oh, they blend in, that's okay. So I'm wondering at this point, actually when you think about like a selection of a model, I mean you basically said that it doesn't matter as long as you have like the proper information architecture, content strategy in place and all. So you would say that there is basically no preference in terms of what you're going for?", "start": 2467.516, "speaker": "SPEAKER_00"}, {"end": 2521.031, "text": "It depends on what you're trying to do so if you want a general high-level Open-ended task a very large foundational model Claude GPT those things is going to be off the shelf what you want because it's post-trained if you want to do a very specific task That benefit of being very broadly trained to do anything is actually a negative because they're not specifically trained to do one particular thing a very particular way.", "start": 2492.808, "speaker": "SPEAKER_02"}, {"end": 2530.596, "text": "Right. So does it mean that then it's in some specific cases like an enterprise world where you want to kind of really optimize a specific task instead of going for large language models to go for small language models?", "start": 2521.231, "speaker": "SPEAKER_00"}, {"end": 2579.644, "text": "The architecture is usually a combination of multiple things. So you use smaller fine-tuned models for very specific tasks where you want to do something concrete. Let's talk about Brad's example earlier. If I want my particular pattern library, to be a very strong influence on the creation of components, then I'm going to create a bunch of training examples and I'm going to fine tune a model that's already good at writing code to write exactly my specific kind of code. But in that same pipeline, I may use something bigger to answer like very high level things like make a plan for what this web page should be. Right, so that is a very broad high-level world knowledge task and then like turn it into this code using this library is a very different kind of specific task I see use a smaller fine-tune model, right?", "start": 2531.176, "speaker": "SPEAKER_02"}, {"end": 2662.552, "text": "I'm also wondering just how come like actually those things like citations, right? And and some of those nice these features that you find that in perplexity and in Claude like just sharing the actual output being able to Reef kind of fork it like what you get into this is fantastic, but most AI experiences are not like that. So I'm actually wondering, I mean, maybe that's my personal experience, but I just want to share and I'm wondering what you think as well. I think over the last maybe three weeks, I have used Google once. Maybe just once. I mean, Google Maps is a different story, right? But I mean, like, like literally searching and going through links, really rare. But at the same time, over the last two months, I maybe used CHGPT once. So the only thing I'm paying for is perplexity, and I'm using it all the time. And I mean, like, all the time. And I get the reference. Perplexity employee. Perplexity? No, no. Perplexity stock owner. But I think for me, the reason why I'm saying that is that, to me, perplexity is doing this incredible job and kind of combining all these different patterns that you were also mentioning. But why is it not more like a norm? Why is the norm still just chatbot? Why are most web pages pieces of shit? Yeah, but, well, if you look at your GPT, right, it's, you know, over two years, it feels like it's better, but not better, if it makes sense.", "start": 2579.884, "speaker": "SPEAKER_00"}, {"end": 2694.176, "text": "My joke was not a joke, really, right? Like, if you look at the early stages of mobile apps, the vast majority of mobile apps at the beginning were moving web pages into a tiny little screen. They were pretty bad. They're trying to apply very prior patterns to a new medium, and it just takes a while to learn how to use that medium, what the experiences should be, and there's going to be more evolution beyond what I showed and what perplexity is doing, whatever. We're still super early in this stuff.", "start": 2664.253, "speaker": "SPEAKER_02"}, {"end": 2732.168, "text": "But at the same time, I feel like there is a bit, I mean, we had a very intense discussion about that yesterday. Because again, when you look at how AI is perceived outside of the US, it's very different. Like in Europe, whenever you start having conversations, there was a conference around you at AI. And then there is this sense of frustration, I guess, which would be the right word. And the reason for that is very simple, because many people are speaking about, you know, 2022 AI comes up, everybody's like, wow, that's magical. Now, everybody's like, when people speak about AI, it's like it feels fake or hallucinating or slow because you always have to wait for your GPT out.", "start": 2694.716, "speaker": "SPEAKER_00"}, {"end": 2734.249, "text": "You can't render hands. What's wrong?", "start": 2732.188, "speaker": "SPEAKER_02"}, {"end": 2751.759, "text": "Yeah, like all those things. But I mean, I'm just wondering, where are we now with the idea? Do we feel like it's like really groundbreaking innovation? Because everybody was expecting, it felt like a big breakthrough in health care and whatnot. Is it where we are? Or is it not? What is the state of it now?", "start": 2734.309, "speaker": "SPEAKER_00"}, {"end": 2755.1, "text": "I mean, I literally showed you a digital version of my brain.", "start": 2751.979, "speaker": "SPEAKER_02"}, {"end": 2759.402, "text": "Which is fantastic to see, but you also have it all documented neatly, I think.", "start": 2755.601, "speaker": "SPEAKER_00"}, {"end": 2764.145, "text": "We should have digital versions of your brain and everybody else's brains.", "start": 2760.223, "speaker": "SPEAKER_02"}, {"end": 2770.003, "text": "And then you have this... Then I'll know what you're doing with these.", "start": 2767.682, "speaker": "SPEAKER_00"}, {"end": 2776.307, "text": "Oh, yeah, yeah, yeah. That's right. That's right. But do you see... No, it's the same answer. We're still so early.", "start": 2770.784, "speaker": "SPEAKER_02"}, {"end": 2776.867, "text": "Yeah.", "start": 2776.687, "speaker": "SPEAKER_01"}, {"end": 2790.895, "text": "We're going to figure this out. Like, when the iPhone came out in 2007, was there a TikTok? Nope. It's like everybody's brain now fused to TikTok in a terrible way. Yeah.", "start": 2776.947, "speaker": "SPEAKER_02"}, {"end": 2793.019, "text": "Right.", "start": 2792.739, "speaker": "SPEAKER_00"}, {"end": 2857.23, "text": "So like those, it's just, you know, there's a ton of time which will put, so if you ask me, I think as I tried to illustrate there, you can basically put AI throughout your whole stack. From the user experience all the way down to how you store information right to like what your data structures are or no data structures whatsoever your data structures are just text and then you can basically build up and down from there like that's. Pretty crazy. Even in all these other previous kind of like major technology changes we had, we didn't have anything that is the entire stack all at once. Parts of the stack changed and that had massive repercussions on the industry. Web, mobile, you name it, cloud, more on the lower side of the stack. But now the whole stack is basically being rethought and changed. So I think the changes that we're gonna see of this, again, personally, I believe are gonna be more dramatic than those.", "start": 2793.079, "speaker": "SPEAKER_02"}, {"end": 2885.932, "text": "Actually, I should probably also cover some questions from the audience, by the way. We can talk about your questions. No, no, that's okay. We can have a conversation anyway. There was a question that came up quickly. Can you talk about content strategy for AI? What does it mean? How do you maintain your application model for not just existing, but, ooh, that just jumped. But new content, yeah. New content and something on that.", "start": 2859.584, "speaker": "SPEAKER_00"}, {"end": 2970.737, "text": "Fascinating topic. I don't think anybody really knows yet. The one thing I didn't show with the Ask Luke system that I can, if people are interested, is when questions come in that I don't have answers to inside my content corpus, but I actually have an answer in my head. I've gotten into this workflow where I built an admin system where I just press a button and talk. It records all that, like I don't have to write a blog article or anything like that, it just slurps it all up and that becomes part of the content corpus and it grows. And so behind the scenes over the past year and a half, I've generated something like 500 equivalent articles, which imagine publishing 500 blog articles in a year. You'd be crazy, right? It doesn't happen. But now, Question comes in, I know what people are asking about, and if I have an answer, I just hit record, talk, don't care about the format, don't care about the process. And so that is a very different way of thinking about content strategy, because it's such a precious thing before, and it's so structured. And now you can, again, basically use natural language as your data structure. I've personally found it a tough bit to flip in my head. For a long time, I was still like, well, I have to then format it into an article, and I have to remove the parts that are... No, you don't.", "start": 2886.292, "speaker": "SPEAKER_02"}, {"end": 2984.226, "text": "Drop it all in there and let the system figure it out. And when you think about this type of data consumption, that's another question that came up and processing. How does it impact document format structures? Are we moving away from PDFs for good?", "start": 2972.355, "speaker": "SPEAKER_00"}, {"end": 2992.334, "text": "No, because people still want PDFs as output, which is funny. One of the requests I have is like, can you turn that answer into a PDF?", "start": 2985.387, "speaker": "SPEAKER_02"}, {"end": 2997.527, "text": "Well, now we can turn it into audio podcast.", "start": 2995.566, "speaker": "SPEAKER_00"}, {"end": 3017.058, "text": "Yeah, turn it into a podcast. It's funny. I always, not always, but I have in the past made fun of Microsoft Copilot because Microsoft Copilot sits alongside Microsoft Office and it's always like, you're making a Word document. Do you want to turn it into a PowerPoint? And then it's like, you're making a PowerPoint. Do you want to turn it into a Word document?", "start": 2997.567, "speaker": "SPEAKER_02"}, {"end": 3036.117, "text": "What am I doing? Right? Pat is wondering how do you test AI for hallucination? We are running a financial application and we can't suggest inaccurate recommendations or insights. Accuracy is very critical. We haven't started working on AI yet because we're afraid of giving wrong suggestions to our users.", "start": 3018.458, "speaker": "SPEAKER_00"}, {"end": 3117.713, "text": "Yep, so the reason why I did this little Ask Luke project year and a half ago or whatever was because of that exact question. It's very difficult to assess, is this right or wrong? When I showed the answer there, where did Luke go to school? I know the answer. Do any of you? No. So how can you tell if the system is right or wrong? And the reason we built that system on my corpus is because I know all that content, and I could assess and verify what is wrong or right. So when you have that capability, then you can train the system to be like, that's the correct answer. That's the wrong answer. That's the correct answer. That's the wrong answer. I literally have an interface that our AI researcher made for me, which is like, vote on the response, which one is correct. And I have sat there for hours being like left, right, left, right, left, right. But that was, you know, a long time ago. Now I think the systems are a lot better. At that, and it's obviously an issue that everybody's aware of in the space. I think it's becoming less and less of a concern. And especially if you do the type of thing that I showed there, right? It's not foolproof, but the odds of things going off the rails are like this as opposed to... like this and it'll get better and better and they'll be more fine tuning that you can do it.", "start": 3036.437, "speaker": "SPEAKER_02"}, {"end": 3126.197, "text": "And so how much time did it take from the very beginning when you just had all the documents to more or less decent results?", "start": 3118.053, "speaker": "SPEAKER_00"}, {"end": 3165.426, "text": "This is the thing about AI. You can very quickly get to a compelling demo that's like 80% or 90% of what you want. Right and we saw a bunch of that today I think in some of the earlier presentations like oh look I type this into Blank and out pops a thing That's kind of what I want the next 10% is like 90% of the work right and that's true in other Domains as well, but it's particularly true in AI to stitch together something. That's working pretty good Super fast, very easy. To make something that's really robust and works for a long time, really long slog, lots of work.", "start": 3126.257, "speaker": "SPEAKER_02"}, {"end": 3193.068, "text": "Maybe final question, then of course we'll have more conversations at the party, I'm sure. Ben, that was expected. Are you worried about artificial general intelligence, AGI? You are? It's not my question. You're answering first. Well, I'm from Europe, we have to. Yes, you are. Don't worry, you've already regulated it. It's fine. Yeah, we have to protect the humanity and all.", "start": 3167.367, "speaker": "SPEAKER_00"}, {"end": 3196.709, "text": "The law has been signed, so it's no longer an issue.", "start": 3193.789, "speaker": "SPEAKER_02"}, {"end": 3208.633, "text": "You're welcome. So what are you thinking? No. What do you mean no? I mean, like, are you worried? Oh, I can't wait. That's a different way of looking at it.", "start": 3196.749, "speaker": "SPEAKER_00"}, {"end": 3214.595, "text": "If there's machines that can do all the crap work that I have to do all day and I can just mountain bike all day, like bring it on.", "start": 3209.033, "speaker": "SPEAKER_02"}, {"end": 3221.068, "text": "All right, well, let's stay hopeful then, I would say. All right, thank you so much, Luc, for being with us.", "start": 3216.112, "speaker": "SPEAKER_00"}], "transcriptByParagraphs": [{"end": 0, "text": "the last session of the day. And we have Luke with us. Look, lurking in the back. Hello, Luke. Hello. So we both are green.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Yes, you finally caught up.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We finally like this is like you match the slides. So did we? Yeah, the color partially. Yeah, there is a bit of that. So AI, huh? Still a thing. I was I think it's a thing. Somebody mentioned it earlier today. Yeah. Yeah. So we've been having conversation about, uh, well, many things yesterday. And then of course the AI came along and, oh my, look is all in, like all in 100% in AI is this. So what do you do? Anybody actually working on a product that now embeds or integrates AI in some way? A few people? Okay. Who has given up on AI? Like AI is like so 2022. Okay.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "A few people, a few people, maybe, maybe look will change your mind today as well. Apparently, AI has eaten, well, looks website. Don't give it away. I should. Okay. I will not. So, but let's see. I mean, again, looking at what we, not, not what we can do with the AI, but maybe what we should do with the AI and how it all works under the hood. And also just for you to know, of course, let me just drop, drop in back in here.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "So Lucas Karanti, Managing Director, Sato Whittle Hill Ventures, but also earlier was a product director at Google and spent a lot of time of his life with little things like web form design. Remember that? Yeah, I do. 2010. Irrelevant now.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "And mobile? Yeah, mobile, also irrelevant.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Well, then, I don't know what to expect now. Well, AI, I guess. So, please, round of applause to Lukro Blewski.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Okay, thank you for having me. And, uh, there we go. And all of my little AI guys. Now, the staff here is so on top of it because we blew up these balloons and the moment you guys walked out the door, they took them all away. And I had an elaborate scheme for all of us stomping on the balloon at the same time and making a big noise. And now that has completely fallen apart. Thank you. But we're going to try this. Can you stand up for a second? You've been sitting really long. Your butt's probably hurt. Your brains are fried.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Okay, so instead of stomping on a balloon, just try to make a big stomp noise. Okay, we have to time it, all right? Ready? So I'm here to talk about how AI ate my website, right? And when we think about that title, you might have some scary thoughts, AI, eating a website. What could that be? That doesn't sound very friendly, right? You might imagine some scary things. Okay, so on the count of three, I'm gonna try to time this. I want you to stomp a really scary stomp, and I'm gonna bring in a very scary AI thing. Okay, you ready? One, two, three. Oh, you did it!", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You conjured it! So here's a giant computer brain eating up all these web pages. Ah! My little guys! Let it go! Anybody scared? Okay, you can sit down. Don't worry, all the little A guys, AI guys, we'll be back. Woo! Okay. So in truth, when you think about AI, by the way, it's not as emphatic as when everybody pops a balloon. But that was good. I like that. Thank you. In truth, though, when most people think about AI, they think about something like chat GPT. We saw a bunch of examples of people using prompt engineering through chat GPT. This is a large language model.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You can ask it questions like, who am I? Who is Luke? So I don't really need an intro slide anymore, because all this stuff is just available. But it's not just my website that these giant AI models have sort of subsumed and eaten up, right? They've also eaten up the entire rest of the web. And so when you ask a question like, where did Luke go to school? It's going to respond to something that seems really likely, right? Because people who do product designs very likely that they actually went to CMU. Probably many of them did. But I did not.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So I say, I think Luke went to UIC and it responds, you're right, I'm so sorry, but he went to grad school at CMU, which I did not. So then I can say, did I really go to CMU? You're like, of course not, you're right. So no problem with reinforcement learning in these models. They're very oriented to doing what you say, right? You're wrong. I apologize. You're wrong again. I apologize. So we can kind of get them into whatever state we want. And sometimes they'll veer off into weird places, which I'm not going to do because the intro to this talk was weird enough.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But let's zoom out for a second and look at this relationship. So we have websites, like my little website, where probably that information about me existed or maybe existed on the SmashingConf website. And you've got lots and lots of these websites, right? And all these things get basically turned into training data for these immense models. And they then can generate a response based on all the things they've learned about. What's the most likely thing to come next? And then they get tuned in different directions with things like reinforcement learning.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So hence why it assumes that I as a product designer went to CMU. But this whole concept of training giant machine brains on the totality of published content on the internet You know, that doesn't sound nice, right? It sounds kind of scary, hence my scary AI thing here, conjures these scary narratives. But it doesn't have to be this way, because if we stop thinking in terms of this monolithic giant AI brain, And we instead start thinking about all the things different AI models, large or small, can do.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And then we start thinking about how they can actually help us create new user experiences, build products in different ways. We heard a little bit about those in the morning sessions. There's a lot of things these things can do with language. They can basically take speech, turn into text. They can take text, turn it into speech. They can transform languages from one to the other. With vision, they can identify objects, identify faces. They can segment stuff. There's this huge list of capabilities that we now have because of these very capable AI models.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And what's different from a few years ago, we have always had, not always, but machine learning's been around for a long time. But the dramatic shift over the past maybe two-ish years is that these very, very capable models are now accessible to anyone, anywhere and everywhere. You can download them, you can run them from the cloud. And so all these capabilities are just flying all over the place and showing up in so many different places. So people are learning about all these capabilities and integrating them into lots of different places.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And this is the way I really like to think about this change, as opposed to the scary AI brain. To me, it's much more like, what are the new ingredients that we currently have that we can use to rethink experiences, to do things we couldn't do before, right? To go from all of these different technologies to new capabilities, especially on the web. Because God bless Brad and his enthusiasm for the original principles of the web. The web is not a very cool place right now.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You go there and you're just inudated with like 15 dialogues and subscriptions for newsletters and you can't even read the content and honestly the first thing you want to do is kind of leave. It's not very cool. But I think we can make it really cool. So with that in mind, I'm just gonna walk you through a whole bunch of details on putting these things to work. And the place where I've been putting them to work is the place that I can control the most, which is my little part of the worldwide web here. This is my website.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "I started this 28 years ago, so how many of you are less than 28 years old? Okay, yeah, that's what I thought. What can you do in 28 years of having a website so you have things to aspire to? Well, I've written 2,000 text articles, put up about 100-something videos, bunch of PDF presentations, made almost 2,000 images. Yes, you too can achieve such things. That's a lot of stuff. And the way you can make use of that stuff using a web browser or the internet today is mostly like two modalities. You can search for things, right? You come back with some results. You can browse around.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But even just my little website has 500,000 written words, 6,000 minutes of video. So like, God bless you if you ever make your way even through part of that. Highly unlikely. But what if we didn't have to just do search and browse? What if we could actually mirror the real world? And the real world that we're in, Vitaly keeps reminding us about the Pac-Man, right? Talk to each other. What if you could just talk? And so you could ask me a question like, hey, Luke, how do I decide what metrics I should use for my app?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And in the real world, what I would do is be like, hmm, what do I know about that? I'll think about maybe a bunch of the things I've written, maybe some of the talks I've given in the past. And then I, in my head, assemble an answer and really quickly respond to you. The human brain is an amazing thing, much more amazing than any of these AI models we have. But I would think about what I know and within, you know, fractions of a second be able to assemble it into a coherent response and just keep talking and answer you. That's pretty different than our options online.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "If you do search, how do I pick metrics from my app? Go fish through these links. If you're browsing around and you make your way to all the articles tagged with metrics, well, there's 243 of them. So again, good luck. But what if instead we can do what I'm saying? How do I pick metrics from my app? So go through everything I've written or said on the topic, put them together into a coherent response, cite the sources, so you can go even deeper or verify what I said. So in this case, this response, how do I pick metrics from my app?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You could see if you put this in the little system, I'm going to show you. You get this answer.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And one of it is coming from a talk called Mind the Gap from 1 Minute 17. one hour 17 minutes and 50 seconds you click on that and it will take you to that part of the video right there queued up and you can see on the slide in the background how do I decide what to measure so those citations are tied directly to where the information is coming from so This is what happens when a bunch of AI models effectively eat all the pieces of content that make up a website and spit up a very different interaction model. And the first question people always have is, well, how do you do that?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Well, I'll get to that. But first, I want to look at what the experience is, and then I'll talk through the pieces of how it gets put together. Now, when you see this, I ask a question, you respond with an answer. Many people say, well, isn't this just a chatbot? What LLM did you use for this chatbot? Is this chat GPT? This is what most people start thinking. Why does chat keep showing up? Number one, chat is a paradigm that's extremely familiar to all of us. You all have probably spent way too much time in your messaging app, even today, some of you may be on it right now.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So this whole paradigm of messaging people, very familiar, very ubiquitous in our lives. So it's quite easy to kind of get in and running with that. But the pattern of this is a series of alternating messages, right? I say something, someone replies, the list goes on, If a message runs long, the space for it, you know, you get what these things look like, right? But is that really a great interaction design pattern for communicating to a global super brain? Maybe. I don't know. Probably not, though.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "If you actually look and see some of the issues that come up with this type of pattern, and this is some research from Nielsen Norman, You know, it's not really optimal for an iterative conversation with a set of documents. So they did a bunch of tests on these large language model-based chat experiences. And in particular, people having to kind of scroll through these conversation threads to find things, remember what was what, kind of veer things in the right direction.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And a quote on the bottom here I think is pretty meaningful, which is, this behavior is a significant point of friction for everybody. that they observed in these studies. So we could probably do a bit better. Now, to account for some of the things that they found in those studies, we made use of a different interaction pattern when creating the experience you saw a second ago. So instead of this back and forth chat, everything expands, what you'll find in this design is something that looks a little bit more like this. And so what does it actually look like?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Basically, all the previous question and answer pairs get collapsed after you've messaged them or completed them. Then the current one that you're on will open up, show you lots of space for the answer. And you can kind of go back and forth within them. So let me just show you what this looks like rather than talking about diagrams. So here you can see, how does free shipping help? It's part of this conversation. There's the response. If I want to go back and see how do you provide real time feedback, then I open up that one. And how does free shipping help thing close up?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So I can kind of access one question and answer pair at a time. And then I can focus on that specific question that I have and its response when I'm asking it. And the rest kind of just rolls up and gets out of the way. So you can much more easily scan, find the thing you were looking for before, and get back to it. And this interaction actually works a little bit more like a FAQ style pattern. You guys have probably seen these. So it's like a big ask thing, right? And then there's all these frequently asked questions that you can open up and expand.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Pretty much most websites have this stuff. And so this is a nice example of when you start from the problem space, right? When you don't start with, oh, we need a chatbot. Let's implement a chatbot. And you think about, well, what are we trying to do? Well, people are asking questions. They're trying to get responses. It's tied to a specific set of information. Maybe there's a different interaction paradigm that makes a little bit more sense here. And this Q&A interface that I just showed you looks a little bit more like a FAQ pattern than a chatbot pattern.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And I bring this up because too often the process of design starts with something like a competitive audit. Or let's go see what everybody else is doing, put it up on a wall, right? And then as soon as you do that, if you've ever ridden a bike and you're riding a bike and you look at a tree, where are you gonna go? You're gonna ride right into the tree, right? It's called target fixation. So as soon as you see that chatbot pattern on chatGPT, guess where your design is gonna go? Right into that pattern.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And whether intentionally or not, you end up copying what these different things are doing instead of letting the problem space actually guide the solution. So in this case, starting with understanding that problem of people are asking questions, they need answers, there's going to be a lot of them, they're going to go back and try and reference things, they need a way to scan through stuff, we ended up a little bit more of a FAQ thing than a chatbot thing. So now there's this expandable conversational interface. You ask a question, response comes down, some citations come up.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But for the purpose of this presentation, because these are big slides on a screen, I've simplified a few things here. So I actually want to show you two more things on how this interface works. And one important one is this concept of suggested content. So you could see here. there's a response and then immediately after the response there's a series of follow-on questions that are related to the question you just asked. The first couple are directly tied to some of the questions that show up there and then the other ones are broader and kind of expand to other things that you can ask.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And these suggested questions are actually really critical in anything that is just this big open-ended Box. Because one of the big problems people have with these things is, what do I ask? I don't know. Well, you can ask anything. That's like saying, don't ask anything. Nobody knows. And so having something like this addresses the issue of what should I ask. And lots of people have that problem because something like 65 or 70% of all the questions that get asked in this interface come from these suggested questions, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So they're very helpful for getting people started and moving them in the right direction. So we knew from the start these were important, but it took a bit to actually iterate through the designs to get to what you saw there. So at first we experimented with something like very explicit to trigger suggested questions, because we're like, oh, we don't want to show these all the time. Somebody should be able to kind of click on something and get those. So we started with this little light bulb, right? Give me an idea, get it?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So you click the light bulb, bunch of suggested questions come up. But then you get the classic, nobody knows what that icon is, are they gonna click it? So then you try something like, what else can I ask? And that shows up. But in both cases, the burden is on people to ask for these things. So instead, here's a series of designs that actually just started dropping it right after every answer, you'd get these suggestions. And there was a couple iterations on this until finally, We ended up on the most obvious way you could possibly do this, which is the way design always works.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's obvious in retrospect. Just put it directly in line right after the answer. Make it look almost like the answers, except a tiny bit different and away you go. So these follow directly in line in the interface. And the other important detail that I'm not showing, which is something we just launched a week ago, maybe two weeks ago, is now we created this new capability and you have to like squint the CDC is little like ones and twos popping up over there. These now are direct citations of the part of the answer that relates to where that content came from.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So you can kind of see within the response which information piece is exciting. And if you click on them, as I'll show you in a second, you can go and see that information. So further kind of making obvious the way this thing works. And the way this works is it's pulling all this information from my website, right? All those articles, all those videos, all those audio files, and using that information to create a bunch of responses. for people's questions about design. Now, if you see one of those little citations, you can click on them and it'll expand it directly in line.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So to go back to my diagram before here, how does free shipping help? You can see it's saying this article on evolving e-commerce shipping was used to make this answer. You can open up and there's the article and you have the ability to ask questions just of that article or go back to this more global mode. So what kinds of things can answer questions is not just those articles. As I mentioned before, there's videos, podcasts, PDFs, images, and more. But how do you get this stuff to be usable in this kind of interface?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You have web pages, they have PDFs, you have websites, they've got images, you've got all this stuff. How do we take those things and then use them in this type of interface? So there's a couple things that we need to do and this is where the ability for you to use a whole bunch of different AI models starts to, I think, emerge. So, for example, if we look at videos, we have like a video, like this talk, let's say, is recorded. There'll be a video file at the end of it. We can use a speech-to-text model to take that video file, turn it into text.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We can use a diarization model to break up who's talking when, when there's the Q&A session. We can have different voices and split that up. We can generate summaries, keywords, and then we can use vision models to extract items in those video screenshots, people, that kind of thing. So we take these files, we run them through a bunch of AI tasks using a number of models, and then we have to do a bunch of cleanup of that content. And when I say cleanup, I mean like we make sure that it understands that Luke is Luke, right? So it kind of cleans up the entities.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We add some metadata, like date, location, et cetera. And so now we have content that we can use to start answering people's questions that before was just a bunch of video files. And to show you how far you can go with this, I'll give you one example. One of the things that we wanted to do was actually get some nice looking thumbnails for these videos. As you can see here, they're kind of all over the place. And so what you can do pretty quickly is generate a bunch of thumbnails from a video.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "This is a pretty trivial script with something like FFMpeg or what have you. spit out a bunch of videos, and then I can manually just go through four or five of those videos, or up to ten or something like that, and pick a good looking thumbnail that I like. And then using that little set of data, here's a bunch of thumbnails, pick this one, here's a bunch of thumbnails, pick this one, as examples, we can fine tune a little AI model that for all the rest of the hundred something videos picks a thumbnail for us.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So the next time it goes through here, it's going to use this lightweight training that we give to pick this beautiful image of me here with the Which Paris Girl Are You screenshot. It nailed that part. It left me with a dumb little grin, but whatever. It's probably the best thumbnail of the bunch. So this is just a quick example of how you can think about having AI models do a lot of things for you. If you can kind of model a behavior to use that word intentionally, and if you can show a machine how to do something, you can train that machine to start doing it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And that, I think, is a really big switch to go off in your head. All right, if I can show a machine how to do this, I can get the machine to do it pretty consistently, even little things like this. And so this opens up a lot of possibilities in terms of, again, how you can invert experiences because you can have machines do all this processing. For audio, we can do the same thing. Take a bunch of podcasts, take a bunch of audio interviews, turn them into text, diarize them, summarize them. Whatever you need, do a bit of processing to them. Again, same kind of thing for articles.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Summarize them, extract keywords, generate suggested questions, all this good stuff. So we start with all of these files that make up your typical web page is, websites, audio, video, text. images, what have you. We use these different AI models to turn them into useful, common kinds of content. And then when we have everything in a common format, this is where some of the secret sauce happens. What we do is we take all those little bits of content, call them chunks, whatever you want to call them, and we basically put them into an embedding space. that puts related concepts to it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "This isn't exactly how it works, but this is a good mental model to have. So, e-commerce-y things go over here. Mobile touchy things are over here. What have you? And in some cases, those are video pieces. In other cases, those are images. In other cases, those are text snippets. So all this stuff is in basically one big, you can think of it as a fuzzy search index or another way to think about it as a semantic search index where you're actually looking things up by their relative meaning as opposed to things like keywords or metadata look up.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So I'll show a system diagram super quick just so folks get it like this always happens. But you start with your docs, you do a whole bunch of this cleaning, you stick them into this search index that organizes them again in a very fuzzy kind of semantic way. And then you need some way of searching against it. You can have multiple ways of searching against it. Back comes some results. You can rank those results. And then you take what you found and you basically use them to generate a response. You can use lots of different models if you want. You can use one model.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You can come up with a number of answers, pick the best one, using another AI model if you get the idea. And then you get to a point where you've got the right information to answer people's questions. So we break all the things into little chunks. We stick them into this crazy vector space. And then when people ask questions, we go find the relevant pieces and assemble them into answers. It's AI models up and down. In fact, it goes even deeper. And hopefully, this is one thing that you take away from this.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "If you go down the rabbit hole, there's like an AI model everywhere in this kind of workflow. Because one of the important things to do is actually learn, right? You get somebody an answer, you want to make sure that your system gets smarter. So one of the things that we did a couple weeks ago is we took all of these lists of questions people ask, and then the list of corresponding documents that we used to answer them, and we trained a custom retriever. which then spits out a score for how good is this document in answering the specific question.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And we did that with using that feedback that we were getting. And if you may say, well, why does that matter? If you just take some off-the-shelf AI model that sticks things into this embedding space, that kind of puts things in the vector space that you can later find it, those are just trained on general sets of sentences, like global internet stuff. So they'll be okay. at finding the pieces that are relevant in my content. But they're not going to be great. And if you make a custom one, these scores mean nothing to you except bigger is better.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So that cross-encoder compared to just using an open AI model does a heck of a lot better. It's much, much better. So you can make your own little AI models that find relevant bits of content, that understand what kind of content you have, and what good content is for the specific kinds of questions that get asked on your sites. And the whole thing becomes a virtuous loop. So some of you may be thinking, oh my god, that's a lot of work. Why would we do this? Why wouldn't I just show GPT on my website? Well, if you ask ChadGPT a question like, why do so many designs look the same?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You're going to get this generic world knowledge answer, right? It's going to be like, well, it's because of cognitive biases, legal and regulatory constraints, economic factors, the influence of trends and cultural influences. It just sounds like a lot of horse shit to me. So all that work I showed you actually allows us to deliver something that mirrors the way I would actually answer a question. Kind of like reflects my specific opinions of things. So instead, when you ask our interface, why do you so many designs look the same?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "I basically say, look, it's because designers are just looking what everybody else is doing and kind of copying it without thinking too much. very different kind of answer. And then the other thing you get is all these citations that I was highlighting earlier, where again, you've got a bunch of information you've published in whatever websites you have or whatever applications you have, you can reference that stuff, make sure that it's cited and allow people to go deeper into all of it. Now, when I say go deeper, it's worth showing a little bit of example of what that actually means.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So we asked this question, why do so many designs look the same? And that answer comes from, I mentioned a few times, videos, audio, web pages, whatever. If you were to click on one of those videos, what we're going to actually do is inside this conversational interface, bring up that video. I showed you an example earlier. Take you to that part of the video where that answer is showing up inside of the video track, and then give you the ability to ask questions of it, give you a.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Summary of that video give you a full transcript that you can search against, so you can click that transcript thing and look for whatever you want inside that video file. If it's an audio file, same thing, we'll just embed an audio player, give you again that transcript and summary, allow you to ask questions of these multi-hour long audio files right then and there. If it is an article, we'll put the article in there. If it's a PDF, same thing, you kind of get it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so again, this is an example of how this interface, this conversational interface, has basically eaten all the stuff that previously made a website. Because now all these things appear in the conversation. You ask a question. Here's the video that answers it. Now you're watching that video. You ask that a question. Here's an audio file. Now you can be inside that audio file or inside that article or through a PDF. Now I said the word PDF and my body sort of cringed a little bit, because PDFs are special, as in very painfully special.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "When I was building this interface, we first started with blog articles, and then we added video, and then we added audio, and then we added PDF. And when we added something like 400-ish PDFs to here, you could see something changed pretty dramatically. So before the question you could ask, what is design? It would say something like, design is the art of gradually applying constraints. So I kind of like this answer over here. And then I added the 400 PDFs, and all of a sudden, This one PDF file from 2006 about dogma free design like just dominated the results.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And now it says it is directed towards human beings with the aim of solving design is the fundamental soul of human creation. I'm like, what is this? This doesn't sound like me. Oh, what turned out is inside this PDF, there was like four screens of quotes from other designers about what is design. And so the model, the information retrieval model is like, hell yeah, look at this baby, right? And it's like. And so the answer just becomes like this regurgitation of a whole bunch of quotes about what is design. So that's not good, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And the other thing that was happening was a lot of junk is kind of showing up as part of this content. So if you break down what's inside, a PDF is like markup language, right? It's got so much stuff in there about how things should look as opposed to what the content is. And so you just get pieces of content that are just crap. I mean, look at some of these things. 1,268, the content is zero nature. One, two, six, five, biography photo, quote, auto video in the press. So you're like, dear AI model, please answer the question with this. And surprising the thing works at all.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's actually a testament to how amazing these things that it works at all. So what we ended up doing is cleaning this up again, getting rid of a bunch of stuff that's crap and clogging up our answers, introducing a mechanism where we diversify the answer. So like, yo, yo, yo, you can't just use this one PDF even though it loves to have design quotes. Break it up a little bit. And when we did that, we actually got to this really nice balance answer for what is design. There's a PDF there, there's a video, there's two podcasts, there's a text file.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And now it's starting to really represent the diversity of content and clean it up. So the answer got a heck of a lot better through cleaning up all this content and balancing what shows up when. And hopefully I'm getting through to you. You know, people ask, like, what chatbot did you use for this? What LLM is this? Is this Chad GPT? Who cares what it is? If you feed it crap, it's going to respond with crap, right? So you really need to take a lot of the document prep and the data pipelines and the cleanup and all this stuff super seriously.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We're dealing in a world now where people are like, well, what am I going to do as a designer? You're going to do data cleanup. You laugh, but it's like, you know, content strategy, anybody remember that? That was like a hot button, I don't know, 20 years ago or something like that. It's a big part of it, and this becomes more like direction, right? Art direction, content direction, model direction, and steering. Somebody needs to have an opinion when I was showing you those two answers on what is design.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Somebody needs to have an opinion on what is the answer that you want to deliver with the system. What should it be? What can't it be? What is not okay? What is okay? Now, we could do even better, because recent advances, and we did this about a month or two ago, in vision models have made it possible for us to totally change the game and what we can do from images and PDFs. So before, we spent a bunch of time trying to get models to do text extraction. We got pretty good at extracting all the real content from a file.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But now what we can do is actually generate things that say something like, the chart illustrates the impact of A-B testing results from 2012 to 2019. The gray line represents this, suggesting a discrepancy between the positive outcomes indicated by A-B testing and actual business results achieved. What do you think is going to make your responses better? This one. Here's another example. Here's a screenshot from my Mind the Gap talk. This is what we had before. Good job getting all that content and image out of there. That's a pretty tough task. Did a good job.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "This is so much better. And so now what we do is we have a pipeline that creates this kind of content from every single screen in the PDF and from every single image. And when you then apply that to your embedding index and you change the content again, you once more have a very different set of answers for what is design. And in this case, I think it's getting better. Like every time I've done this, this answer has gotten better. Every time we've enriched the content and spent the time to make sure that what we're feeding in and how we're processing it is a lot better.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So this is a process of constant adjustment. It's the same thing like people say, OK, I launched the website. I'm done. No, you're not. That's the beginning of it. Oh, I added a chat bot. I'm done. No, that's the beginning of it. So for PDFs, we need lots of cleaning, but now you can see that there's these, in addition to some of the text-based tasks, we've actually got some vision tasks to make sense of all that information and make it useful for these models.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "The other really cool thing that this enables is once you start processing images and understanding semantically what's in an image, you can actually start to respond with images and get multimodal. So now your website doesn't have to just like spit out text. It can actually do things like this. So I ask a question, what's the difference between AR and VR? And like literally, this image shows the spectrum from VR to AR.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You can see that it creates a completely digital world while AR, it's not only putting the image in the right place in the response, but it's actually referencing it in its response. So this creates a whole new set of capabilities. Now you can start to respond with pages of PDFs, too, in the same way. It knows what's in that page of the PDF because we've spent all this time to process it and put it in the embedding index. We've trained a retriever to find it when it's most relevant, and then we can start to assemble answers that make a ton more sense. Okay, that's a lot of stuff.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But hopefully, big picture, let's take a step back. You have website. Website have stuff in it. Yes? Okay, you all know website have stuff. Different kind of stuff. AI models on stuff. Very cool. Whole new ball game. And I think we're just scratching at the surface of this. By the way, this is why I have these little AI guys, because they're hustling through all these little AI models, trying to figure out what to do and how to use what thing, when, and how. Behind the scenes, they're using those language models, those vision models, everything.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so the end result here is not some... Hold on. And the end result here is not some... Scary AI thing, but instead it's a new way to use all this content, right? To kind of mirror real world experiences. You don't have to go through those 10 blue links and click on them and then find the part of the article that kind of relates to the thing that you're trying to go browse through long lists of things. You can just say, what you're asking, right? And so you could say something like, hey, what should designers do about AI? Yeah, it's going to reference some talks.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's going to give you an answer. And away you go. So these conversational interfaces, whatever we want to call them, or whatever they become in the future. I launched this service on my website. By the way, you can all, I should have said this again. You can all go play with it. Ask.LukeW.com. Go ahead, ask whatever questions you want. Don't crash it. But we've been doing this for almost a year and a half now. And like every couple months, there's some big interesting change where we can do new things. Now we're doing these inline citations. Now we're putting images in there.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And the models are actually referencing the images. Next up, we're going to try doing that within inline video files. Who knows? Lots of stuff to do. So on behalf of me and all my little AI guys, Thank you. And I have one question for you. Are we going to actually find out what the bananas are for? Because they've been sitting there all day, and I'm just dying to know. OK. No, no, no. Thank you. That's fine.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "All right.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "They're still here.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "That's right. Toby, you want a banana? I mean, I'm not going to throw bananas at you. That's a bit disrespectful, I think. Thank you so much, Luke, for the insights. Can we switch? Oh, sure.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Am I in your chair? No, I just feel... I have to sit near the bananas after that comment.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Oh, this is my spot. Oh, they blend in, that's okay. So I'm wondering at this point, actually when you think about like a selection of a model, I mean you basically said that it doesn't matter as long as you have like the proper information architecture, content strategy in place and all. So you would say that there is basically no preference in terms of what you're going for?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "It depends on what you're trying to do so if you want a general high-level Open-ended task a very large foundational model Claude GPT those things is going to be off the shelf what you want because it's post-trained if you want to do a very specific task That benefit of being very broadly trained to do anything is actually a negative because they're not specifically trained to do one particular thing a very particular way.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Right. So does it mean that then it's in some specific cases like an enterprise world where you want to kind of really optimize a specific task instead of going for large language models to go for small language models?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "The architecture is usually a combination of multiple things. So you use smaller fine-tuned models for very specific tasks where you want to do something concrete. Let's talk about Brad's example earlier. If I want my particular pattern library, to be a very strong influence on the creation of components, then I'm going to create a bunch of training examples and I'm going to fine tune a model that's already good at writing code to write exactly my specific kind of code.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But in that same pipeline, I may use something bigger to answer like very high level things like make a plan for what this web page should be. Right, so that is a very broad high-level world knowledge task and then like turn it into this code using this library is a very different kind of specific task I see use a smaller fine-tune model, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "I'm also wondering just how come like actually those things like citations, right? And and some of those nice these features that you find that in perplexity and in Claude like just sharing the actual output being able to Reef kind of fork it like what you get into this is fantastic, but most AI experiences are not like that. So I'm actually wondering, I mean, maybe that's my personal experience, but I just want to share and I'm wondering what you think as well. I think over the last maybe three weeks, I have used Google once. Maybe just once.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "I mean, Google Maps is a different story, right? But I mean, like, like literally searching and going through links, really rare. But at the same time, over the last two months, I maybe used CHGPT once. So the only thing I'm paying for is perplexity, and I'm using it all the time. And I mean, like, all the time. And I get the reference. Perplexity employee. Perplexity? No, no. Perplexity stock owner.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "But I think for me, the reason why I'm saying that is that, to me, perplexity is doing this incredible job and kind of combining all these different patterns that you were also mentioning. But why is it not more like a norm? Why is the norm still just chatbot? Why are most web pages pieces of shit? Yeah, but, well, if you look at your GPT, right, it's, you know, over two years, it feels like it's better, but not better, if it makes sense.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "My joke was not a joke, really, right? Like, if you look at the early stages of mobile apps, the vast majority of mobile apps at the beginning were moving web pages into a tiny little screen. They were pretty bad. They're trying to apply very prior patterns to a new medium, and it just takes a while to learn how to use that medium, what the experiences should be, and there's going to be more evolution beyond what I showed and what perplexity is doing, whatever. We're still super early in this stuff.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But at the same time, I feel like there is a bit, I mean, we had a very intense discussion about that yesterday. Because again, when you look at how AI is perceived outside of the US, it's very different. Like in Europe, whenever you start having conversations, there was a conference around you at AI. And then there is this sense of frustration, I guess, which would be the right word. And the reason for that is very simple, because many people are speaking about, you know, 2022 AI comes up, everybody's like, wow, that's magical.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Now, everybody's like, when people speak about AI, it's like it feels fake or hallucinating or slow because you always have to wait for your GPT out.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "You can't render hands. What's wrong?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Yeah, like all those things. But I mean, I'm just wondering, where are we now with the idea? Do we feel like it's like really groundbreaking innovation? Because everybody was expecting, it felt like a big breakthrough in health care and whatnot. Is it where we are? Or is it not? What is the state of it now?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "I mean, I literally showed you a digital version of my brain.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Which is fantastic to see, but you also have it all documented neatly, I think.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "We should have digital versions of your brain and everybody else's brains.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And then you have this... Then I'll know what you're doing with these.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Oh, yeah, yeah, yeah. That's right. That's right. But do you see... No, it's the same answer. We're still so early.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Yeah.", "start": 0, "speaker": "SPEAKER_01"}, {"end": 0, "text": "We're going to figure this out. Like, when the iPhone came out in 2007, was there a TikTok? Nope. It's like everybody's brain now fused to TikTok in a terrible way. Yeah.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Right.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "So like those, it's just, you know, there's a ton of time which will put, so if you ask me, I think as I tried to illustrate there, you can basically put AI throughout your whole stack. From the user experience all the way down to how you store information right to like what your data structures are or no data structures whatsoever your data structures are just text and then you can basically build up and down from there like that's. Pretty crazy. Even in all these other previous kind of like major technology changes we had, we didn't have anything that is the entire stack all at once.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Parts of the stack changed and that had massive repercussions on the industry. Web, mobile, you name it, cloud, more on the lower side of the stack. But now the whole stack is basically being rethought and changed. So I think the changes that we're gonna see of this, again, personally, I believe are gonna be more dramatic than those.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Actually, I should probably also cover some questions from the audience, by the way. We can talk about your questions. No, no, that's okay. We can have a conversation anyway. There was a question that came up quickly. Can you talk about content strategy for AI? What does it mean? How do you maintain your application model for not just existing, but, ooh, that just jumped. But new content, yeah. New content and something on that.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Fascinating topic. I don't think anybody really knows yet. The one thing I didn't show with the Ask Luke system that I can, if people are interested, is when questions come in that I don't have answers to inside my content corpus, but I actually have an answer in my head. I've gotten into this workflow where I built an admin system where I just press a button and talk. It records all that, like I don't have to write a blog article or anything like that, it just slurps it all up and that becomes part of the content corpus and it grows.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so behind the scenes over the past year and a half, I've generated something like 500 equivalent articles, which imagine publishing 500 blog articles in a year. You'd be crazy, right? It doesn't happen. But now, Question comes in, I know what people are asking about, and if I have an answer, I just hit record, talk, don't care about the format, don't care about the process. And so that is a very different way of thinking about content strategy, because it's such a precious thing before, and it's so structured.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And now you can, again, basically use natural language as your data structure. I've personally found it a tough bit to flip in my head. For a long time, I was still like, well, I have to then format it into an article, and I have to remove the parts that are... No, you don't.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Drop it all in there and let the system figure it out. And when you think about this type of data consumption, that's another question that came up and processing. How does it impact document format structures? Are we moving away from PDFs for good?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "No, because people still want PDFs as output, which is funny. One of the requests I have is like, can you turn that answer into a PDF?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Well, now we can turn it into audio podcast.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Yeah, turn it into a podcast. It's funny. I always, not always, but I have in the past made fun of Microsoft Copilot because Microsoft Copilot sits alongside Microsoft Office and it's always like, you're making a Word document. Do you want to turn it into a PowerPoint? And then it's like, you're making a PowerPoint. Do you want to turn it into a Word document?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "What am I doing? Right? Pat is wondering how do you test AI for hallucination? We are running a financial application and we can't suggest inaccurate recommendations or insights. Accuracy is very critical. We haven't started working on AI yet because we're afraid of giving wrong suggestions to our users.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Yep, so the reason why I did this little Ask Luke project year and a half ago or whatever was because of that exact question. It's very difficult to assess, is this right or wrong? When I showed the answer there, where did Luke go to school? I know the answer. Do any of you? No. So how can you tell if the system is right or wrong? And the reason we built that system on my corpus is because I know all that content, and I could assess and verify what is wrong or right. So when you have that capability, then you can train the system to be like, that's the correct answer.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "That's the wrong answer. That's the correct answer. That's the wrong answer. I literally have an interface that our AI researcher made for me, which is like, vote on the response, which one is correct. And I have sat there for hours being like left, right, left, right, left, right. But that was, you know, a long time ago. Now I think the systems are a lot better. At that, and it's obviously an issue that everybody's aware of in the space. I think it's becoming less and less of a concern. And especially if you do the type of thing that I showed there, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's not foolproof, but the odds of things going off the rails are like this as opposed to... like this and it'll get better and better and they'll be more fine tuning that you can do it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so how much time did it take from the very beginning when you just had all the documents to more or less decent results?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "This is the thing about AI. You can very quickly get to a compelling demo that's like 80% or 90% of what you want. Right and we saw a bunch of that today I think in some of the earlier presentations like oh look I type this into Blank and out pops a thing That's kind of what I want the next 10% is like 90% of the work right and that's true in other Domains as well, but it's particularly true in AI to stitch together something. That's working pretty good Super fast, very easy. To make something that's really robust and works for a long time, really long slog, lots of work.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Maybe final question, then of course we'll have more conversations at the party, I'm sure. Ben, that was expected. Are you worried about artificial general intelligence, AGI? You are? It's not my question. You're answering first. Well, I'm from Europe, we have to. Yes, you are. Don't worry, you've already regulated it. It's fine. Yeah, we have to protect the humanity and all.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "The law has been signed, so it's no longer an issue.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You're welcome. So what are you thinking? No. What do you mean no? I mean, like, are you worried? Oh, I can't wait. That's a different way of looking at it.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "If there's machines that can do all the crap work that I have to do all day and I can just mountain bike all day, like bring it on.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "All right, well, let's stay hopeful then, I would say. All right, thank you so much, Luc, for being with us.", "start": 0, "speaker": "SPEAKER_00"}]}, {"id": "9e728f85-426f-4055-bf2a-7434ec08a366", "url": "https://lukew_img.s3.amazonaws.com/video/smashingconfOct2024_howaiatemywebsite720.mov", "date": 0, "type": 2, "title": "Discussing the Integration and Future Potential of AI in Products and Websites", "topics": ["AI integration", "Product development", "Web form design", "Mobile technology", "AI discussion", "AI models", "Conversational interfaces", "Content cleanup", "Data pipelines", "Website design", "Language models", "Vision models", "Inline citations", "Inline video files", "AI content strategy", "AI application models", "AI in healthcare", "AI testing for accuracy", "Artificial General Intelligence (AGI)"], "endTime": 3070.953, "nameMap": {"undefined": "Luke W", "SPEAKER_00": "Luke Wroblewski", "SPEAKER_01": "Speaker A", "SPEAKER_02": "Luc"}, "summary": "The conversation is about the integration and relevance of Artificial Intelligence (AI) in products and websites. It discusses the use of AI models to process different types of content and answer user queries in a personalized and efficient manner. The conversation also touches on the continuous adjustment of AI models, the transformation of traditional website interfaces into interactive ones, and the future potential of AI in technology. The speakers, Luke Wroblewski and Luc, share their experiences and views on the development and testing of AI systems.", "filename": "149e49b5-1b4c-44d0-8af9-9be24815d005-smashingconfOct2024_howaiatemywebsite720", "startTime": 2975.098, "coverImage": "/fallback-audio-cover.png", "transcript": [{"end": 1.591, "text": " the last session of the day.", "start": 0.009, "speaker": "SPEAKER_00"}, {"end": 2.913, "text": "And we have Luke with us.", "start": 2.072, "speaker": "SPEAKER_00"}, {"end": 4.976, "text": "Look, lurking in the back.", "start": 2.953, "speaker": "SPEAKER_00"}, {"end": 6.719, "text": "Hello, Luke.", "start": 6.138, "speaker": "SPEAKER_00"}, {"end": 8.422, "text": "Hello.", "start": 7.981, "speaker": "SPEAKER_00"}, {"end": 10.384, "text": "So we both are green.", "start": 9.443, "speaker": "SPEAKER_00"}, {"end": 12.696, "text": " Yes, you finally caught up.", "start": 11.536, "speaker": "SPEAKER_02"}, {"end": 14.677, "text": "We finally like this is like you match the slides.", "start": 12.716, "speaker": "SPEAKER_00"}, {"end": 15.657, "text": "So did we?", "start": 14.917, "speaker": "SPEAKER_00"}, {"end": 17.577, "text": "Yeah, the color partially.", "start": 16.017, "speaker": "SPEAKER_00"}, {"end": 18.537, "text": "Yeah, there is a bit of that.", "start": 17.597, "speaker": "SPEAKER_00"}, {"end": 20.498, "text": "So AI, huh?", "start": 19.177, "speaker": "SPEAKER_00"}, {"end": 21.718, "text": "Still a thing.", "start": 21.258, "speaker": "SPEAKER_00"}, {"end": 22.938, "text": "I was I think it's a thing.", "start": 21.998, "speaker": "SPEAKER_00"}, {"end": 24.478, "text": "Somebody mentioned it earlier today.", "start": 23.058, "speaker": "SPEAKER_00"}, {"end": 24.918, "text": "Yeah.", "start": 24.618, "speaker": "SPEAKER_00"}, {"end": 25.178, "text": "Yeah.", "start": 24.978, "speaker": "SPEAKER_00"}, {"end": 28.479, "text": "So we've been having conversation about, uh, well, many things yesterday.", "start": 25.218, "speaker": "SPEAKER_00"}, {"end": 35.06, "text": "And then of course the AI came along and, oh my, look is all in, like all in 100% in AI is this.", "start": 28.499, "speaker": "SPEAKER_00"}, {"end": 36.56, "text": "So what do you do?", "start": 35.82, "speaker": "SPEAKER_00"}, {"end": 40.841, "text": "Anybody actually working on a product that now embeds or integrates AI in some way?", "start": 36.62, "speaker": "SPEAKER_00"}, {"end": 42.831, "text": " A few people?", "start": 42.31, "speaker": "SPEAKER_00"}, {"end": 43.631, "text": "Okay.", "start": 42.851, "speaker": "SPEAKER_00"}, {"end": 45.073, "text": "Who has given up on AI?", "start": 43.892, "speaker": "SPEAKER_00"}, {"end": 46.194, "text": "Like AI is like so 2022.", "start": 45.093, "speaker": "SPEAKER_00"}, {"end": 46.494, "text": "Okay.", "start": 46.294, "speaker": "SPEAKER_00"}, {"end": 53.061, "text": "A few people, a few people, maybe, maybe look will change your mind today as well.", "start": 46.554, "speaker": "SPEAKER_00"}, {"end": 58.266, "text": "Apparently, AI has eaten, well, looks website.", "start": 53.401, "speaker": "SPEAKER_00"}, {"end": 59.366, "text": " Don't give it away.", "start": 58.766, "speaker": "SPEAKER_00"}, {"end": 60.207, "text": "I should.", "start": 59.907, "speaker": "SPEAKER_00"}, {"end": 60.627, "text": "Okay.", "start": 60.367, "speaker": "SPEAKER_00"}, {"end": 61.107, "text": "I will not.", "start": 60.707, "speaker": "SPEAKER_00"}, {"end": 62.228, "text": "So, but let's see.", "start": 61.147, "speaker": "SPEAKER_00"}, {"end": 69.172, "text": "I mean, again, looking at what we, not, not what we can do with the AI, but maybe what we should do with the AI and how it all works under the hood.", "start": 62.268, "speaker": "SPEAKER_00"}, {"end": 73.754, "text": "And also just for you to know, of course, let me just drop, drop in back in here.", "start": 69.652, "speaker": "SPEAKER_00"}, {"end": 85.021, "text": "So Lucas Karanti, Managing Director, Sato Whittle Hill Ventures, but also earlier was a product director at Google and spent a lot of time of his life with little things like web form design.", "start": 74.475, "speaker": "SPEAKER_00"}, {"end": 85.681, "text": "Remember that?", "start": 85.061, "speaker": "SPEAKER_00"}, {"end": 86.221, "text": "Yeah, I do.", "start": 85.721, "speaker": "SPEAKER_00"}, {"end": 86.221, "text": "2010.", "start": 86.241, "speaker": "SPEAKER_00"}, {"end": 86.742, "text": "Irrelevant now.", "start": 86.281, "speaker": "SPEAKER_00"}, {"end": 89.921, "text": " And mobile?", "start": 89.241, "speaker": "SPEAKER_02"}, {"end": 91.682, "text": "Yeah, mobile, also irrelevant.", "start": 90.221, "speaker": "SPEAKER_02"}, {"end": 96.063, "text": "Well, then, I don't know what to expect now.", "start": 93.082, "speaker": "SPEAKER_00"}, {"end": 97.704, "text": "Well, AI, I guess.", "start": 96.223, "speaker": "SPEAKER_00"}, {"end": 100.024, "text": "So, please, round of applause to Lukro Blewski.", "start": 98.104, "speaker": "SPEAKER_00"}, {"end": 106.987, "text": "Okay, thank you for having me.", "start": 100.044, "speaker": "SPEAKER_02"}, {"end": 111.168, "text": "And, uh, there we go.", "start": 108.667, "speaker": "SPEAKER_02"}, {"end": 114.069, "text": "And all of my little AI guys.", "start": 112.568, "speaker": "SPEAKER_02"}, {"end": 124.572, "text": " Now, the staff here is so on top of it because we blew up these balloons and the moment you guys walked out the door, they took them all away.", "start": 116.747, "speaker": "SPEAKER_02"}, {"end": 132.138, "text": "And I had an elaborate scheme for all of us stomping on the balloon at the same time and making a big noise.", "start": 125.613, "speaker": "SPEAKER_02"}, {"end": 134.479, "text": "And now that has completely fallen apart.", "start": 132.998, "speaker": "SPEAKER_02"}, {"end": 134.8, "text": "Thank you.", "start": 134.499, "speaker": "SPEAKER_02"}, {"end": 137.122, "text": " But we're going to try this.", "start": 135.6, "speaker": "SPEAKER_02"}, {"end": 138.203, "text": "Can you stand up for a second?", "start": 137.202, "speaker": "SPEAKER_02"}, {"end": 139.665, "text": "You've been sitting really long.", "start": 138.223, "speaker": "SPEAKER_02"}, {"end": 140.766, "text": "Your butt's probably hurt.", "start": 139.705, "speaker": "SPEAKER_02"}, {"end": 142.308, "text": "Your brains are fried.", "start": 141.246, "speaker": "SPEAKER_02"}, {"end": 148.274, "text": "Okay, so instead of stomping on a balloon, just try to make a big stomp noise.", "start": 142.328, "speaker": "SPEAKER_02"}, {"end": 149.976, "text": "Okay, we have to time it, all right?", "start": 148.575, "speaker": "SPEAKER_02"}, {"end": 151.017, "text": "Ready?", "start": 150.777, "speaker": "SPEAKER_02"}, {"end": 154.701, "text": "So I'm here to talk about how AI ate my website, right?", "start": 151.618, "speaker": "SPEAKER_02"}, {"end": 163.339, "text": " And when we think about that title, you might have some scary thoughts, AI, eating a website.", "start": 156.014, "speaker": "SPEAKER_02"}, {"end": 164.78, "text": "What could that be?", "start": 164.16, "speaker": "SPEAKER_02"}, {"end": 166.361, "text": "That doesn't sound very friendly, right?", "start": 164.82, "speaker": "SPEAKER_02"}, {"end": 168.122, "text": "You might imagine some scary things.", "start": 166.381, "speaker": "SPEAKER_02"}, {"end": 171.384, "text": "Okay, so on the count of three, I'm gonna try to time this.", "start": 169.023, "speaker": "SPEAKER_02"}, {"end": 177.008, "text": "I want you to stomp a really scary stomp, and I'm gonna bring in a very scary AI thing.", "start": 171.424, "speaker": "SPEAKER_02"}, {"end": 178.449, "text": "Okay, you ready?", "start": 177.689, "speaker": "SPEAKER_02"}, {"end": 179.61, "text": "One, two, three.", "start": 178.469, "speaker": "SPEAKER_02"}, {"end": 185.242, "text": " Oh, you did it!", "start": 184.622, "speaker": "SPEAKER_02"}, {"end": 187.163, "text": "You conjured it!", "start": 185.442, "speaker": "SPEAKER_02"}, {"end": 192.045, "text": "So here's a giant computer brain eating up all these web pages.", "start": 188.104, "speaker": "SPEAKER_02"}, {"end": 192.265, "text": "Ah!", "start": 192.165, "speaker": "SPEAKER_02"}, {"end": 193.306, "text": "My little guys!", "start": 192.665, "speaker": "SPEAKER_02"}, {"end": 194.466, "text": "Let it go!", "start": 193.986, "speaker": "SPEAKER_02"}, {"end": 197.828, "text": "Anybody scared?", "start": 194.486, "speaker": "SPEAKER_02"}, {"end": 198.728, "text": "Okay, you can sit down.", "start": 198.008, "speaker": "SPEAKER_02"}, {"end": 203.37, "text": "Don't worry, all the little A guys, AI guys, we'll be back.", "start": 200.429, "speaker": "SPEAKER_02"}, {"end": 206.671, "text": "Woo!", "start": 206.291, "speaker": "SPEAKER_02"}, {"end": 207.512, "text": "Okay.", "start": 207.231, "speaker": "SPEAKER_02"}, {"end": 214.815, "text": " So in truth, when you think about AI, by the way, it's not as emphatic as when everybody pops a balloon.", "start": 209.033, "speaker": "SPEAKER_02"}, {"end": 215.356, "text": "But that was good.", "start": 214.835, "speaker": "SPEAKER_02"}, {"end": 215.916, "text": "I like that.", "start": 215.496, "speaker": "SPEAKER_02"}, {"end": 216.276, "text": "Thank you.", "start": 215.956, "speaker": "SPEAKER_02"}, {"end": 222.819, "text": "In truth, though, when most people think about AI, they think about something like chat GPT.", "start": 218.577, "speaker": "SPEAKER_02"}, {"end": 227.42, "text": "We saw a bunch of examples of people using prompt engineering through chat GPT.", "start": 222.859, "speaker": "SPEAKER_02"}, {"end": 229.681, "text": "This is a large language model.", "start": 227.44, "speaker": "SPEAKER_02"}, {"end": 231.662, "text": "You can ask it questions like, who am I?", "start": 229.741, "speaker": "SPEAKER_02"}, {"end": 232.302, "text": "Who is Luke?", "start": 231.802, "speaker": "SPEAKER_02"}, {"end": 237.985, "text": "So I don't really need an intro slide anymore, because all this stuff is just available.", "start": 233.223, "speaker": "SPEAKER_02"}, {"end": 246.629, "text": " But it's not just my website that these giant AI models have sort of subsumed and eaten up, right?", "start": 239.786, "speaker": "SPEAKER_02"}, {"end": 250.071, "text": "They've also eaten up the entire rest of the web.", "start": 246.669, "speaker": "SPEAKER_02"}, {"end": 253.473, "text": "And so when you ask a question like, where did Luke go to school?", "start": 250.572, "speaker": "SPEAKER_02"}, {"end": 257.075, "text": "It's going to respond to something that seems really likely, right?", "start": 254.294, "speaker": "SPEAKER_02"}, {"end": 260.797, "text": "Because people who do product designs very likely that they actually went to CMU.", "start": 257.095, "speaker": "SPEAKER_02"}, {"end": 262.118, "text": "Probably many of them did.", "start": 261.137, "speaker": "SPEAKER_02"}, {"end": 263.719, "text": "But I did not.", "start": 262.858, "speaker": "SPEAKER_02"}, {"end": 273.335, "text": " So I say, I think Luke went to UIC and it responds, you're right, I'm so sorry, but he went to grad school at CMU, which I did not.", "start": 265.669, "speaker": "SPEAKER_02"}, {"end": 277.278, "text": "So then I can say, did I really go to CMU?", "start": 274.115, "speaker": "SPEAKER_02"}, {"end": 278.739, "text": "You're like, of course not, you're right.", "start": 277.298, "speaker": "SPEAKER_02"}, {"end": 283.709, "text": " So no problem with reinforcement learning in these models.", "start": 280.948, "speaker": "SPEAKER_02"}, {"end": 287.891, "text": "They're very oriented to doing what you say, right?", "start": 283.749, "speaker": "SPEAKER_02"}, {"end": 288.631, "text": "You're wrong.", "start": 288.251, "speaker": "SPEAKER_02"}, {"end": 289.731, "text": "I apologize.", "start": 288.991, "speaker": "SPEAKER_02"}, {"end": 290.372, "text": "You're wrong again.", "start": 289.811, "speaker": "SPEAKER_02"}, {"end": 291.132, "text": "I apologize.", "start": 290.432, "speaker": "SPEAKER_02"}, {"end": 294.393, "text": "So we can kind of get them into whatever state we want.", "start": 291.492, "speaker": "SPEAKER_02"}, {"end": 299.956, "text": "And sometimes they'll veer off into weird places, which I'm not going to do because the intro to this talk was weird enough.", "start": 294.413, "speaker": "SPEAKER_02"}, {"end": 304.844, "text": " But let's zoom out for a second and look at this relationship.", "start": 302.403, "speaker": "SPEAKER_02"}, {"end": 311.826, "text": "So we have websites, like my little website, where probably that information about me existed or maybe existed on the SmashingConf website.", "start": 304.864, "speaker": "SPEAKER_02"}, {"end": 315.107, "text": "And you've got lots and lots of these websites, right?", "start": 312.586, "speaker": "SPEAKER_02"}, {"end": 321.229, "text": "And all these things get basically turned into training data for these immense models.", "start": 315.467, "speaker": "SPEAKER_02"}, {"end": 326.451, "text": "And they then can generate a response based on all the things they've learned about.", "start": 321.889, "speaker": "SPEAKER_02"}, {"end": 328.812, "text": "What's the most likely thing to come next?", "start": 326.491, "speaker": "SPEAKER_02"}, {"end": 334.474, "text": " And then they get tuned in different directions with things like reinforcement learning.", "start": 330.092, "speaker": "SPEAKER_02"}, {"end": 340.537, "text": "So hence why it assumes that I as a product designer went to CMU.", "start": 335.014, "speaker": "SPEAKER_02"}, {"end": 352.523, "text": "But this whole concept of training giant machine brains on the totality of published content on the internet", "start": 341.618, "speaker": "SPEAKER_02"}, {"end": 354.884, "text": " You know, that doesn't sound nice, right?", "start": 353.263, "speaker": "SPEAKER_02"}, {"end": 360.187, "text": "It sounds kind of scary, hence my scary AI thing here, conjures these scary narratives.", "start": 354.904, "speaker": "SPEAKER_02"}, {"end": 368.031, "text": "But it doesn't have to be this way, because if we stop thinking in terms of this monolithic giant AI brain,", "start": 360.867, "speaker": "SPEAKER_02"}, {"end": 374.379, "text": " And we instead start thinking about all the things different AI models, large or small, can do.", "start": 369.011, "speaker": "SPEAKER_02"}, {"end": 381.649, "text": "And then we start thinking about how they can actually help us create new user experiences, build products in different ways.", "start": 374.98, "speaker": "SPEAKER_02"}, {"end": 384.333, "text": "We heard a little bit about those in the morning sessions.", "start": 381.689, "speaker": "SPEAKER_02"}, {"end": 387.577, "text": " There's a lot of things these things can do with language.", "start": 385.356, "speaker": "SPEAKER_02"}, {"end": 390.877, "text": "They can basically take speech, turn into text.", "start": 388.297, "speaker": "SPEAKER_02"}, {"end": 393.638, "text": "They can take text, turn it into speech.", "start": 391.337, "speaker": "SPEAKER_02"}, {"end": 396.339, "text": "They can transform languages from one to the other.", "start": 393.738, "speaker": "SPEAKER_02"}, {"end": 400.14, "text": "With vision, they can identify objects, identify faces.", "start": 396.379, "speaker": "SPEAKER_02"}, {"end": 401.78, "text": "They can segment stuff.", "start": 400.6, "speaker": "SPEAKER_02"}, {"end": 408.822, "text": "There's this huge list of capabilities that we now have because of these very capable AI models.", "start": 402.22, "speaker": "SPEAKER_02"}, {"end": 414.323, "text": "And what's different from a few years ago, we have always had, not always, but machine learning's been around for a long time.", "start": 408.862, "speaker": "SPEAKER_02"}, {"end": 423.426, "text": " But the dramatic shift over the past maybe two-ish years is that these very, very capable models are now accessible to anyone, anywhere and everywhere.", "start": 414.983, "speaker": "SPEAKER_02"}, {"end": 426.387, "text": "You can download them, you can run them from the cloud.", "start": 423.826, "speaker": "SPEAKER_02"}, {"end": 432.63, "text": "And so all these capabilities are just flying all over the place and showing up in so many different places.", "start": 426.828, "speaker": "SPEAKER_02"}, {"end": 437.652, "text": "So people are learning about all these capabilities and integrating them into lots of different places.", "start": 433.09, "speaker": "SPEAKER_02"}, {"end": 444.041, "text": " And this is the way I really like to think about this change, as opposed to the scary AI brain.", "start": 438.737, "speaker": "SPEAKER_02"}, {"end": 454.127, "text": "To me, it's much more like, what are the new ingredients that we currently have that we can use to rethink experiences, to do things we couldn't do before, right?", "start": 444.681, "speaker": "SPEAKER_02"}, {"end": 460.952, "text": "To go from all of these different technologies to new capabilities, especially on the web.", "start": 454.508, "speaker": "SPEAKER_02"}, {"end": 465.835, "text": "Because God bless Brad and his enthusiasm for the original principles of the web.", "start": 461.492, "speaker": "SPEAKER_02"}, {"end": 468.157, "text": "The web is not a very cool place right now.", "start": 466.095, "speaker": "SPEAKER_02"}, {"end": 478.011, "text": " You go there and you're just inudated with like 15 dialogues and subscriptions for newsletters and you can't even read the content and honestly the first thing you want to do is kind of leave.", "start": 469.058, "speaker": "SPEAKER_02"}, {"end": 480.534, "text": "It's not very cool.", "start": 478.972, "speaker": "SPEAKER_02"}, {"end": 483.959, "text": "But I think we can make it really cool.", "start": 482.177, "speaker": "SPEAKER_02"}, {"end": 490.751, "text": " So with that in mind, I'm just gonna walk you through a whole bunch of details on putting these things to work.", "start": 484.986, "speaker": "SPEAKER_02"}, {"end": 498.158, "text": "And the place where I've been putting them to work is the place that I can control the most, which is my little part of the worldwide web here.", "start": 491.572, "speaker": "SPEAKER_02"}, {"end": 499.039, "text": "This is my website.", "start": 498.178, "speaker": "SPEAKER_02"}, {"end": 504.684, "text": "I started this 28 years ago, so how many of you are less than 28 years old?", "start": 499.799, "speaker": "SPEAKER_02"}, {"end": 507.166, "text": "Okay, yeah, that's what I thought.", "start": 505.945, "speaker": "SPEAKER_02"}, {"end": 513.533, "text": " What can you do in 28 years of having a website so you have things to aspire to?", "start": 509.451, "speaker": "SPEAKER_02"}, {"end": 522.298, "text": "Well, I've written 2,000 text articles, put up about 100-something videos, bunch of PDF presentations, made almost 2,000 images.", "start": 513.593, "speaker": "SPEAKER_02"}, {"end": 524.319, "text": "Yes, you too can achieve such things.", "start": 522.398, "speaker": "SPEAKER_02"}, {"end": 528.301, "text": "That's a lot of stuff.", "start": 527.481, "speaker": "SPEAKER_02"}, {"end": 531.863, "text": "And the way you can make use of that stuff", "start": 529.782, "speaker": "SPEAKER_02"}, {"end": 536.943, "text": " using a web browser or the internet today is mostly like two modalities.", "start": 533.599, "speaker": "SPEAKER_02"}, {"end": 538.185, "text": "You can search for things, right?", "start": 536.963, "speaker": "SPEAKER_02"}, {"end": 540.127, "text": "You come back with some results.", "start": 538.205, "speaker": "SPEAKER_02"}, {"end": 541.168, "text": "You can browse around.", "start": 540.167, "speaker": "SPEAKER_02"}, {"end": 547.355, "text": "But even just my little website has 500,000 written words, 6,000 minutes of video.", "start": 541.849, "speaker": "SPEAKER_02"}, {"end": 551.56, "text": "So like, God bless you if you ever make your way even through part of that.", "start": 548.176, "speaker": "SPEAKER_02"}, {"end": 553.34, "text": " Highly unlikely.", "start": 552.54, "speaker": "SPEAKER_02"}, {"end": 557.942, "text": "But what if we didn't have to just do search and browse?", "start": 554.501, "speaker": "SPEAKER_02"}, {"end": 559.822, "text": "What if we could actually mirror the real world?", "start": 557.982, "speaker": "SPEAKER_02"}, {"end": 564.444, "text": "And the real world that we're in, Vitaly keeps reminding us about the Pac-Man, right?", "start": 560.643, "speaker": "SPEAKER_02"}, {"end": 565.284, "text": "Talk to each other.", "start": 564.504, "speaker": "SPEAKER_02"}, {"end": 567.265, "text": "What if you could just talk?", "start": 566.024, "speaker": "SPEAKER_02"}, {"end": 574.067, "text": "And so you could ask me a question like, hey, Luke, how do I decide what metrics I should use for my app?", "start": 568.205, "speaker": "SPEAKER_02"}, {"end": 580.252, "text": " And in the real world, what I would do is be like, hmm, what do I know about that?", "start": 575.129, "speaker": "SPEAKER_02"}, {"end": 584.816, "text": "I'll think about maybe a bunch of the things I've written, maybe some of the talks I've given in the past.", "start": 580.273, "speaker": "SPEAKER_02"}, {"end": 590.139, "text": "And then I, in my head, assemble an answer and really quickly respond to you.", "start": 585.356, "speaker": "SPEAKER_02"}, {"end": 595.202, "text": "The human brain is an amazing thing, much more amazing than any of these AI models we have.", "start": 590.159, "speaker": "SPEAKER_02"}, {"end": 604.509, "text": "But I would think about what I know and within, you know, fractions of a second be able to assemble it into a coherent response and just keep talking and answer you.", "start": 595.242, "speaker": "SPEAKER_02"}, {"end": 607.691, "text": " That's pretty different than our options online.", "start": 605.149, "speaker": "SPEAKER_02"}, {"end": 611.354, "text": "If you do search, how do I pick metrics from my app?", "start": 608.452, "speaker": "SPEAKER_02"}, {"end": 613.396, "text": "Go fish through these links.", "start": 611.434, "speaker": "SPEAKER_02"}, {"end": 620.662, "text": "If you're browsing around and you make your way to all the articles tagged with metrics, well, there's 243 of them.", "start": 614.637, "speaker": "SPEAKER_02"}, {"end": 622.544, "text": "So again, good luck.", "start": 620.722, "speaker": "SPEAKER_02"}, {"end": 626.527, "text": "But what if instead we can do what I'm saying?", "start": 624.385, "speaker": "SPEAKER_02"}, {"end": 628.529, "text": "How do I pick metrics from my app?", "start": 626.747, "speaker": "SPEAKER_02"}, {"end": 637.743, "text": " So go through everything I've written or said on the topic, put them together into a coherent response, cite the sources, so you can go even deeper or verify what I said.", "start": 629.639, "speaker": "SPEAKER_02"}, {"end": 641.344, "text": "So in this case, this response, how do I pick metrics from my app?", "start": 638.623, "speaker": "SPEAKER_02"}, {"end": 645.186, "text": "You could see if you put this in the little system, I'm going to show you.", "start": 641.925, "speaker": "SPEAKER_02"}, {"end": 646.266, "text": "You get this answer.", "start": 645.386, "speaker": "SPEAKER_02"}, {"end": 650.188, "text": "And one of it is coming from a talk called Mind the Gap from 1 Minute 17.", "start": 646.827, "speaker": "SPEAKER_02"}, {"end": 671.697, "text": " one hour 17 minutes and 50 seconds you click on that and it will take you to that part of the video right there queued up and you can see on the slide in the background how do I decide what to measure so those citations are tied directly to where the information is coming from so", "start": 652.189, "speaker": "SPEAKER_02"}, {"end": 681.738, "text": " This is what happens when a bunch of AI models effectively eat all the pieces of content that make up a website and spit up a very different interaction model.", "start": 672.935, "speaker": "SPEAKER_02"}, {"end": 685.8, "text": "And the first question people always have is, well, how do you do that?", "start": 682.839, "speaker": "SPEAKER_02"}, {"end": 686.82, "text": "Well, I'll get to that.", "start": 685.9, "speaker": "SPEAKER_02"}, {"end": 691.922, "text": "But first, I want to look at what the experience is, and then I'll talk through the pieces of how it gets put together.", "start": 686.86, "speaker": "SPEAKER_02"}, {"end": 695.403, "text": "Now, when you see this, I ask a question, you respond with an answer.", "start": 692.502, "speaker": "SPEAKER_02"}, {"end": 697.804, "text": "Many people say, well, isn't this just a chatbot?", "start": 695.463, "speaker": "SPEAKER_02"}, {"end": 702.585, "text": " What LLM did you use for this chatbot?", "start": 699.783, "speaker": "SPEAKER_02"}, {"end": 703.706, "text": "Is this chat GPT?", "start": 702.765, "speaker": "SPEAKER_02"}, {"end": 706.228, "text": "This is what most people start thinking.", "start": 704.727, "speaker": "SPEAKER_02"}, {"end": 711.261, "text": " Why does chat keep showing up?", "start": 708.76, "speaker": "SPEAKER_02"}, {"end": 715.263, "text": "Number one, chat is a paradigm that's extremely familiar to all of us.", "start": 711.301, "speaker": "SPEAKER_02"}, {"end": 721.786, "text": "You all have probably spent way too much time in your messaging app, even today, some of you may be on it right now.", "start": 716.083, "speaker": "SPEAKER_02"}, {"end": 729.269, "text": "So this whole paradigm of messaging people, very familiar, very ubiquitous in our lives.", "start": 722.206, "speaker": "SPEAKER_02"}, {"end": 731.97, "text": "So it's quite easy to kind of get in and running with that.", "start": 729.309, "speaker": "SPEAKER_02"}, {"end": 735.412, "text": "But the pattern of this is a series of alternating messages, right?", "start": 732.01, "speaker": "SPEAKER_02"}, {"end": 738.113, "text": "I say something, someone replies, the list goes on,", "start": 735.492, "speaker": "SPEAKER_02"}, {"end": 742.957, "text": " If a message runs long, the space for it, you know, you get what these things look like, right?", "start": 738.593, "speaker": "SPEAKER_02"}, {"end": 751.284, "text": "But is that really a great interaction design pattern for communicating to a global super brain?", "start": 744.118, "speaker": "SPEAKER_02"}, {"end": 752.905, "text": "Maybe.", "start": 752.645, "speaker": "SPEAKER_02"}, {"end": 753.746, "text": "I don't know.", "start": 752.925, "speaker": "SPEAKER_02"}, {"end": 755.147, "text": "Probably not, though.", "start": 754.507, "speaker": "SPEAKER_02"}, {"end": 763.574, "text": "If you actually look and see some of the issues that come up with this type of pattern, and this is some research from Nielsen Norman,", "start": 756.468, "speaker": "SPEAKER_02"}, {"end": 769.103, "text": " You know, it's not really optimal for an iterative conversation with a set of documents.", "start": 764.38, "speaker": "SPEAKER_02"}, {"end": 773.687, "text": "So they did a bunch of tests on these large language model-based chat experiences.", "start": 769.524, "speaker": "SPEAKER_02"}, {"end": 782.873, "text": "And in particular, people having to kind of scroll through these conversation threads to find things, remember what was what, kind of veer things in the right direction.", "start": 774.347, "speaker": "SPEAKER_02"}, {"end": 790.779, "text": "And a quote on the bottom here I think is pretty meaningful, which is, this behavior is a significant point of friction for everybody.", "start": 782.893, "speaker": "SPEAKER_02"}, {"end": 792.96, "text": " that they observed in these studies.", "start": 791.559, "speaker": "SPEAKER_02"}, {"end": 794.661, "text": "So we could probably do a bit better.", "start": 793.18, "speaker": "SPEAKER_02"}, {"end": 806.63, "text": "Now, to account for some of the things that they found in those studies, we made use of a different interaction pattern when creating the experience you saw a second ago.", "start": 796.082, "speaker": "SPEAKER_02"}, {"end": 815.556, "text": "So instead of this back and forth chat, everything expands, what you'll find in this design is something that looks a little bit more like this.", "start": 806.65, "speaker": "SPEAKER_02"}, {"end": 819.218, "text": " And so what does it actually look like?", "start": 817.335, "speaker": "SPEAKER_02"}, {"end": 827.991, "text": "Basically, all the previous question and answer pairs get collapsed after you've messaged them or completed them.", "start": 819.538, "speaker": "SPEAKER_02"}, {"end": 832.458, "text": "Then the current one that you're on will open up, show you lots of space for the answer.", "start": 828.151, "speaker": "SPEAKER_02"}, {"end": 836.3, "text": " And you can kind of go back and forth within them.", "start": 833.599, "speaker": "SPEAKER_02"}, {"end": 838.702, "text": "So let me just show you what this looks like rather than talking about diagrams.", "start": 836.32, "speaker": "SPEAKER_02"}, {"end": 841.183, "text": "So here you can see, how does free shipping help?", "start": 838.722, "speaker": "SPEAKER_02"}, {"end": 842.643, "text": "It's part of this conversation.", "start": 841.263, "speaker": "SPEAKER_02"}, {"end": 844.024, "text": "There's the response.", "start": 843.124, "speaker": "SPEAKER_02"}, {"end": 849.426, "text": "If I want to go back and see how do you provide real time feedback, then I open up that one.", "start": 844.124, "speaker": "SPEAKER_02"}, {"end": 852.168, "text": "And how does free shipping help thing close up?", "start": 849.527, "speaker": "SPEAKER_02"}, {"end": 857.711, "text": " So I can kind of access one question and answer pair at a time.", "start": 853.068, "speaker": "SPEAKER_02"}, {"end": 864.255, "text": "And then I can focus on that specific question that I have and its response when I'm asking it.", "start": 858.331, "speaker": "SPEAKER_02"}, {"end": 867.236, "text": "And the rest kind of just rolls up and gets out of the way.", "start": 864.295, "speaker": "SPEAKER_02"}, {"end": 871.139, "text": "So you can much more easily scan, find the thing you were looking for before, and get back to it.", "start": 867.276, "speaker": "SPEAKER_02"}, {"end": 876.982, "text": "And this interaction actually works a little bit more like a FAQ style pattern.", "start": 872.419, "speaker": "SPEAKER_02"}, {"end": 878.263, "text": "You guys have probably seen these.", "start": 877.022, "speaker": "SPEAKER_02"}, {"end": 883.721, "text": " So it's like a big ask thing, right?", "start": 880.318, "speaker": "SPEAKER_02"}, {"end": 887.644, "text": "And then there's all these frequently asked questions that you can open up and expand.", "start": 883.741, "speaker": "SPEAKER_02"}, {"end": 889.626, "text": "Pretty much most websites have this stuff.", "start": 887.844, "speaker": "SPEAKER_02"}, {"end": 895.759, "text": " And so this is a nice example of when you start from the problem space, right?", "start": 890.956, "speaker": "SPEAKER_02"}, {"end": 897.561, "text": "When you don't start with, oh, we need a chatbot.", "start": 895.779, "speaker": "SPEAKER_02"}, {"end": 899.062, "text": "Let's implement a chatbot.", "start": 897.761, "speaker": "SPEAKER_02"}, {"end": 900.743, "text": "And you think about, well, what are we trying to do?", "start": 899.182, "speaker": "SPEAKER_02"}, {"end": 902.664, "text": "Well, people are asking questions.", "start": 901.263, "speaker": "SPEAKER_02"}, {"end": 904.285, "text": "They're trying to get responses.", "start": 902.704, "speaker": "SPEAKER_02"}, {"end": 907.207, "text": "It's tied to a specific set of information.", "start": 904.345, "speaker": "SPEAKER_02"}, {"end": 911.23, "text": "Maybe there's a different interaction paradigm that makes a little bit more sense here.", "start": 907.607, "speaker": "SPEAKER_02"}, {"end": 917.454, "text": "And this Q&A interface that I just showed you looks a little bit more like a FAQ pattern than a chatbot pattern.", "start": 912.31, "speaker": "SPEAKER_02"}, {"end": 925.802, "text": " And I bring this up because too often the process of design starts with something like a competitive audit.", "start": 918.7, "speaker": "SPEAKER_02"}, {"end": 929.843, "text": "Or let's go see what everybody else is doing, put it up on a wall, right?", "start": 926.322, "speaker": "SPEAKER_02"}, {"end": 936.625, "text": "And then as soon as you do that, if you've ever ridden a bike and you're riding a bike and you look at a tree, where are you gonna go?", "start": 930.543, "speaker": "SPEAKER_02"}, {"end": 939.426, "text": "You're gonna ride right into the tree, right?", "start": 937.685, "speaker": "SPEAKER_02"}, {"end": 941.306, "text": "It's called target fixation.", "start": 940.006, "speaker": "SPEAKER_02"}, {"end": 945.387, "text": "So as soon as you see that chatbot pattern on chatGPT, guess where your design is gonna go?", "start": 941.326, "speaker": "SPEAKER_02"}, {"end": 947.388, "text": "Right into that pattern.", "start": 946.448, "speaker": "SPEAKER_02"}, {"end": 959.887, "text": " And whether intentionally or not, you end up copying what these different things are doing instead of letting the problem space actually guide the solution.", "start": 948.641, "speaker": "SPEAKER_02"}, {"end": 975.056, "text": "So in this case, starting with understanding that problem of people are asking questions, they need answers, there's going to be a lot of them, they're going to go back and try and reference things, they need a way to scan through stuff, we ended up a little bit more of a FAQ thing than a chatbot thing.", "start": 960.788, "speaker": "SPEAKER_02"}, {"end": 980.247, "text": " So now there's this expandable conversational interface.", "start": 976.485, "speaker": "SPEAKER_02"}, {"end": 984.208, "text": "You ask a question, response comes down, some citations come up.", "start": 980.267, "speaker": "SPEAKER_02"}, {"end": 990.731, "text": "But for the purpose of this presentation, because these are big slides on a screen, I've simplified a few things here.", "start": 985.149, "speaker": "SPEAKER_02"}, {"end": 994.673, "text": "So I actually want to show you two more things on how this interface works.", "start": 990.771, "speaker": "SPEAKER_02"}, {"end": 999.055, "text": "And one important one is this concept of suggested content.", "start": 995.533, "speaker": "SPEAKER_02"}, {"end": 1001.816, "text": "So you could see here.", "start": 1000.395, "speaker": "SPEAKER_02"}, {"end": 1010.125, "text": " there's a response and then immediately after the response there's a series of follow-on questions that are related to the question you just asked.", "start": 1003.24, "speaker": "SPEAKER_02"}, {"end": 1019.951, "text": "The first couple are directly tied to some of the questions that show up there and then the other ones are broader and kind of expand to other things that you can ask.", "start": 1010.745, "speaker": "SPEAKER_02"}, {"end": 1026.496, "text": "And these suggested questions are actually really critical in anything that is just this big open-ended", "start": 1021.192, "speaker": "SPEAKER_02"}, {"end": 1028.075, "text": " Box.", "start": 1027.694, "speaker": "SPEAKER_02"}, {"end": 1031.74, "text": "Because one of the big problems people have with these things is, what do I ask?", "start": 1028.876, "speaker": "SPEAKER_02"}, {"end": 1032.201, "text": "I don't know.", "start": 1031.84, "speaker": "SPEAKER_02"}, {"end": 1034.083, "text": "Well, you can ask anything.", "start": 1033.102, "speaker": "SPEAKER_02"}, {"end": 1036.627, "text": "That's like saying, don't ask anything.", "start": 1034.905, "speaker": "SPEAKER_02"}, {"end": 1037.749, "text": "Nobody knows.", "start": 1037.188, "speaker": "SPEAKER_02"}, {"end": 1041.559, "text": " And so having something like this addresses the issue of what should I ask.", "start": 1038.737, "speaker": "SPEAKER_02"}, {"end": 1053.588, "text": "And lots of people have that problem because something like 65 or 70% of all the questions that get asked in this interface come from these suggested questions, right?", "start": 1041.599, "speaker": "SPEAKER_02"}, {"end": 1057.23, "text": "So they're very helpful for getting people started and moving them in the right direction.", "start": 1053.688, "speaker": "SPEAKER_02"}, {"end": 1065.556, "text": "So we knew from the start these were important, but it took a bit to actually iterate through the designs to get to what you saw there.", "start": 1058.191, "speaker": "SPEAKER_02"}, {"end": 1072.284, "text": " So at first we experimented with something like very explicit to trigger suggested questions, because we're like, oh, we don't want to show these all the time.", "start": 1066.477, "speaker": "SPEAKER_02"}, {"end": 1076.45, "text": "Somebody should be able to kind of click on something and get those.", "start": 1072.344, "speaker": "SPEAKER_02"}, {"end": 1078.072, "text": "So we started with this little light bulb, right?", "start": 1076.47, "speaker": "SPEAKER_02"}, {"end": 1079.293, "text": "Give me an idea, get it?", "start": 1078.352, "speaker": "SPEAKER_02"}, {"end": 1083.478, "text": "So you click the light bulb, bunch of suggested questions come up.", "start": 1079.373, "speaker": "SPEAKER_02"}, {"end": 1088.579, "text": " But then you get the classic, nobody knows what that icon is, are they gonna click it?", "start": 1084.517, "speaker": "SPEAKER_02"}, {"end": 1091.481, "text": "So then you try something like, what else can I ask?", "start": 1088.619, "speaker": "SPEAKER_02"}, {"end": 1092.381, "text": "And that shows up.", "start": 1091.541, "speaker": "SPEAKER_02"}, {"end": 1097.684, "text": "But in both cases, the burden is on people to ask for these things.", "start": 1093.422, "speaker": "SPEAKER_02"}, {"end": 1107.709, "text": "So instead, here's a series of designs that actually just started dropping it right after every answer, you'd get these suggestions.", "start": 1098.624, "speaker": "SPEAKER_02"}, {"end": 1111.711, "text": "And there was a couple iterations on this until finally,", "start": 1108.369, "speaker": "SPEAKER_02"}, {"end": 1118.268, "text": " We ended up on the most obvious way you could possibly do this, which is the way design always works.", "start": 1113.464, "speaker": "SPEAKER_02"}, {"end": 1119.569, "text": "It's obvious in retrospect.", "start": 1118.348, "speaker": "SPEAKER_02"}, {"end": 1122.791, "text": "Just put it directly in line right after the answer.", "start": 1119.989, "speaker": "SPEAKER_02"}, {"end": 1126.975, "text": "Make it look almost like the answers, except a tiny bit different and away you go.", "start": 1122.872, "speaker": "SPEAKER_02"}, {"end": 1133.88, "text": "So these follow directly in line in the interface.", "start": 1129.277, "speaker": "SPEAKER_02"}, {"end": 1141.707, "text": "And the other important detail that I'm not showing, which is something we just launched a week ago, maybe two weeks ago,", "start": 1134.761, "speaker": "SPEAKER_02"}, {"end": 1149.963, "text": " is now we created this new capability and you have to like squint the CDC is little like ones and twos popping up over there.", "start": 1142.894, "speaker": "SPEAKER_02"}, {"end": 1160.216, "text": "These now are direct citations of the part of the answer that relates to where that content came from.", "start": 1151.024, "speaker": "SPEAKER_02"}, {"end": 1165.771, "text": " So you can kind of see within the response which information piece is exciting.", "start": 1161.468, "speaker": "SPEAKER_02"}, {"end": 1170.555, "text": "And if you click on them, as I'll show you in a second, you can go and see that information.", "start": 1165.951, "speaker": "SPEAKER_02"}, {"end": 1175.579, "text": "So further kind of making obvious the way this thing works.", "start": 1171.556, "speaker": "SPEAKER_02"}, {"end": 1180.582, "text": "And the way this works is it's pulling all this information from my website, right?", "start": 1176.399, "speaker": "SPEAKER_02"}, {"end": 1187.948, "text": "All those articles, all those videos, all those audio files, and using that information to create a bunch of responses.", "start": 1180.642, "speaker": "SPEAKER_02"}, {"end": 1191.251, "text": " for people's questions about design.", "start": 1189.71, "speaker": "SPEAKER_02"}, {"end": 1198.016, "text": "Now, if you see one of those little citations, you can click on them and it'll expand it directly in line.", "start": 1192.272, "speaker": "SPEAKER_02"}, {"end": 1202.198, "text": "So to go back to my diagram before here, how does free shipping help?", "start": 1198.216, "speaker": "SPEAKER_02"}, {"end": 1208.502, "text": "You can see it's saying this article on evolving e-commerce shipping was used to make this answer.", "start": 1202.418, "speaker": "SPEAKER_02"}, {"end": 1215.687, "text": "You can open up and there's the article and you have the ability to ask questions just of that article or go back to this more global mode.", "start": 1208.562, "speaker": "SPEAKER_02"}, {"end": 1221.319, "text": " So what kinds of things can answer questions is not just those articles.", "start": 1217.915, "speaker": "SPEAKER_02"}, {"end": 1225.324, "text": "As I mentioned before, there's videos, podcasts, PDFs, images, and more.", "start": 1221.539, "speaker": "SPEAKER_02"}, {"end": 1231.572, "text": "But how do you get this stuff to be usable in this kind of interface?", "start": 1226.766, "speaker": "SPEAKER_02"}, {"end": 1238.479, "text": " You have web pages, they have PDFs, you have websites, they've got images, you've got all this stuff.", "start": 1233.956, "speaker": "SPEAKER_02"}, {"end": 1243.422, "text": "How do we take those things and then use them in this type of interface?", "start": 1238.519, "speaker": "SPEAKER_02"}, {"end": 1254.009, "text": "So there's a couple things that we need to do and this is where the ability for you to use a whole bunch of different AI models starts to, I think, emerge.", "start": 1244.223, "speaker": "SPEAKER_02"}, {"end": 1259.471, "text": " So, for example, if we look at videos, we have like a video, like this talk, let's say, is recorded.", "start": 1254.549, "speaker": "SPEAKER_02"}, {"end": 1261.371, "text": "There'll be a video file at the end of it.", "start": 1259.791, "speaker": "SPEAKER_02"}, {"end": 1267.473, "text": "We can use a speech-to-text model to take that video file, turn it into text.", "start": 1261.871, "speaker": "SPEAKER_02"}, {"end": 1273.015, "text": "We can use a diarization model to break up who's talking when, when there's the Q&A session.", "start": 1268.013, "speaker": "SPEAKER_02"}, {"end": 1275.916, "text": "We can have different voices and split that up.", "start": 1273.575, "speaker": "SPEAKER_02"}, {"end": 1285.668, "text": " We can generate summaries, keywords, and then we can use vision models to extract items in those video screenshots, people, that kind of thing.", "start": 1276.696, "speaker": "SPEAKER_02"}, {"end": 1295.456, "text": " So we take these files, we run them through a bunch of AI tasks using a number of models, and then we have to do a bunch of cleanup of that content.", "start": 1286.673, "speaker": "SPEAKER_02"}, {"end": 1301.778, "text": "And when I say cleanup, I mean like we make sure that it understands that Luke is Luke, right?", "start": 1296.256, "speaker": "SPEAKER_02"}, {"end": 1303.819, "text": "So it kind of cleans up the entities.", "start": 1301.998, "speaker": "SPEAKER_02"}, {"end": 1308.14, "text": "We add some metadata, like date, location, et cetera.", "start": 1303.859, "speaker": "SPEAKER_02"}, {"end": 1309.961, "text": "And so now we have", "start": 1308.86, "speaker": "SPEAKER_02"}, {"end": 1316.203, "text": " content that we can use to start answering people's questions that before was just a bunch of video files.", "start": 1311.141, "speaker": "SPEAKER_02"}, {"end": 1322.005, "text": "And to show you how far you can go with this, I'll give you one example.", "start": 1317.383, "speaker": "SPEAKER_02"}, {"end": 1327.186, "text": "One of the things that we wanted to do was actually get some nice looking thumbnails for these videos.", "start": 1322.065, "speaker": "SPEAKER_02"}, {"end": 1329.467, "text": "As you can see here, they're kind of all over the place.", "start": 1327.527, "speaker": "SPEAKER_02"}, {"end": 1336.169, "text": "And so what you can do pretty quickly is generate a bunch of thumbnails from a video.", "start": 1330.408, "speaker": "SPEAKER_02"}, {"end": 1339.611, "text": "This is a pretty trivial script with something like FFMpeg or what have you.", "start": 1336.189, "speaker": "SPEAKER_02"}, {"end": 1351.576, "text": " spit out a bunch of videos, and then I can manually just go through four or five of those videos, or up to ten or something like that, and pick a good looking thumbnail that I like.", "start": 1340.951, "speaker": "SPEAKER_02"}, {"end": 1363.762, "text": "And then using that little set of data, here's a bunch of thumbnails, pick this one, here's a bunch of thumbnails, pick this one, as examples, we can fine tune a little AI model that for all the rest of the hundred something videos picks a thumbnail for us.", "start": 1352.437, "speaker": "SPEAKER_02"}, {"end": 1375.545, "text": " So the next time it goes through here, it's going to use this lightweight training that we give to pick this beautiful image of me here with the Which Paris Girl Are You screenshot.", "start": 1365.031, "speaker": "SPEAKER_02"}, {"end": 1381.751, "text": " It nailed that part.", "start": 1380.911, "speaker": "SPEAKER_02"}, {"end": 1383.632, "text": "It left me with a dumb little grin, but whatever.", "start": 1381.771, "speaker": "SPEAKER_02"}, {"end": 1385.992, "text": "It's probably the best thumbnail of the bunch.", "start": 1384.312, "speaker": "SPEAKER_02"}, {"end": 1391.893, "text": "So this is just a quick example of how you can think about having AI models do a lot of things for you.", "start": 1386.412, "speaker": "SPEAKER_02"}, {"end": 1401.835, "text": "If you can kind of model a behavior to use that word intentionally, and if you can show a machine how to do something, you can train that machine to start doing it.", "start": 1392.573, "speaker": "SPEAKER_02"}, {"end": 1405.616, "text": "And that, I think, is a really big switch to go off in your head.", "start": 1402.435, "speaker": "SPEAKER_02"}, {"end": 1414.799, "text": " All right, if I can show a machine how to do this, I can get the machine to do it pretty consistently, even little things like this.", "start": 1407.213, "speaker": "SPEAKER_02"}, {"end": 1424.026, "text": "And so this opens up a lot of possibilities in terms of, again, how you can invert experiences because you can have machines do all this processing.", "start": 1416.42, "speaker": "SPEAKER_02"}, {"end": 1425.607, "text": "For audio, we can do the same thing.", "start": 1424.046, "speaker": "SPEAKER_02"}, {"end": 1433.013, "text": "Take a bunch of podcasts, take a bunch of audio interviews, turn them into text, diarize them, summarize them.", "start": 1426.147, "speaker": "SPEAKER_02"}, {"end": 1437.289, "text": " Whatever you need, do a bit of processing to them.", "start": 1434.508, "speaker": "SPEAKER_02"}, {"end": 1441.81, "text": "Again, same kind of thing for articles.", "start": 1439.329, "speaker": "SPEAKER_02"}, {"end": 1447.231, "text": "Summarize them, extract keywords, generate suggested questions, all this good stuff.", "start": 1442.89, "speaker": "SPEAKER_02"}, {"end": 1455.933, "text": "So we start with all of these files that make up your typical web page is, websites, audio, video, text.", "start": 1447.711, "speaker": "SPEAKER_02"}, {"end": 1458.194, "text": " images, what have you.", "start": 1457.333, "speaker": "SPEAKER_02"}, {"end": 1463.62, "text": "We use these different AI models to turn them into useful, common kinds of content.", "start": 1458.755, "speaker": "SPEAKER_02"}, {"end": 1468.845, "text": "And then when we have everything in a common format, this is where some of the secret sauce happens.", "start": 1464.04, "speaker": "SPEAKER_02"}, {"end": 1476.374, "text": "What we do is we take all those little bits of content, call them chunks, whatever you want to call them, and we basically put them into an embedding space.", "start": 1468.886, "speaker": "SPEAKER_02"}, {"end": 1478.856, "text": " that puts related concepts to it.", "start": 1476.974, "speaker": "SPEAKER_02"}, {"end": 1481.779, "text": "This isn't exactly how it works, but this is a good mental model to have.", "start": 1478.876, "speaker": "SPEAKER_02"}, {"end": 1484.522, "text": "So, e-commerce-y things go over here.", "start": 1481.839, "speaker": "SPEAKER_02"}, {"end": 1486.885, "text": "Mobile touchy things are over here.", "start": 1485.023, "speaker": "SPEAKER_02"}, {"end": 1487.566, "text": "What have you?", "start": 1487.105, "speaker": "SPEAKER_02"}, {"end": 1489.968, "text": "And in some cases, those are video pieces.", "start": 1487.586, "speaker": "SPEAKER_02"}, {"end": 1491.25, "text": "In other cases, those are images.", "start": 1489.988, "speaker": "SPEAKER_02"}, {"end": 1493.012, "text": "In other cases, those are text snippets.", "start": 1491.31, "speaker": "SPEAKER_02"}, {"end": 1510.79, "text": " So all this stuff is in basically one big, you can think of it as a fuzzy search index or another way to think about it as a semantic search index where you're actually looking things up by their relative meaning as opposed to things like keywords or metadata look up.", "start": 1495.374, "speaker": "SPEAKER_02"}, {"end": 1517.683, "text": " So I'll show a system diagram super quick just so folks get it like this always happens.", "start": 1513.14, "speaker": "SPEAKER_02"}, {"end": 1527.729, "text": "But you start with your docs, you do a whole bunch of this cleaning, you stick them into this search index that organizes them again in a very fuzzy kind of semantic way.", "start": 1517.723, "speaker": "SPEAKER_02"}, {"end": 1530.39, "text": "And then you need some way of searching against it.", "start": 1528.169, "speaker": "SPEAKER_02"}, {"end": 1533.372, "text": "You can have multiple ways of searching against it.", "start": 1531.451, "speaker": "SPEAKER_02"}, {"end": 1535.851, "text": " Back comes some results.", "start": 1534.611, "speaker": "SPEAKER_02"}, {"end": 1537.112, "text": "You can rank those results.", "start": 1535.911, "speaker": "SPEAKER_02"}, {"end": 1541.613, "text": "And then you take what you found and you basically use them to generate a response.", "start": 1537.712, "speaker": "SPEAKER_02"}, {"end": 1544.413, "text": "You can use lots of different models if you want.", "start": 1542.393, "speaker": "SPEAKER_02"}, {"end": 1545.473, "text": "You can use one model.", "start": 1544.453, "speaker": "SPEAKER_02"}, {"end": 1551.035, "text": "You can come up with a number of answers, pick the best one, using another AI model if you get the idea.", "start": 1546.073, "speaker": "SPEAKER_02"}, {"end": 1557.156, "text": "And then you get to a point where you've got the right information to answer people's questions.", "start": 1551.695, "speaker": "SPEAKER_02"}, {"end": 1559.765, "text": " So we break all the things into little chunks.", "start": 1557.864, "speaker": "SPEAKER_02"}, {"end": 1563.327, "text": "We stick them into this crazy vector space.", "start": 1560.225, "speaker": "SPEAKER_02"}, {"end": 1569.23, "text": "And then when people ask questions, we go find the relevant pieces and assemble them into answers.", "start": 1564.087, "speaker": "SPEAKER_02"}, {"end": 1572.932, "text": "It's AI models up and down.", "start": 1570.351, "speaker": "SPEAKER_02"}, {"end": 1574.633, "text": "In fact, it goes even deeper.", "start": 1573.412, "speaker": "SPEAKER_02"}, {"end": 1578.335, "text": "And hopefully, this is one thing that you take away from this.", "start": 1574.873, "speaker": "SPEAKER_02"}, {"end": 1583.558, "text": "If you go down the rabbit hole, there's like an AI model everywhere in this kind of workflow.", "start": 1578.375, "speaker": "SPEAKER_02"}, {"end": 1587.02, "text": " Because one of the important things to do is actually learn, right?", "start": 1584.158, "speaker": "SPEAKER_02"}, {"end": 1589.982, "text": "You get somebody an answer, you want to make sure that your system gets smarter.", "start": 1587.06, "speaker": "SPEAKER_02"}, {"end": 1605.973, "text": "So one of the things that we did a couple weeks ago is we took all of these lists of questions people ask, and then the list of corresponding documents that we used to answer them, and we trained a custom retriever.", "start": 1591.683, "speaker": "SPEAKER_02"}, {"end": 1611.667, "text": " which then spits out a score for how good is this document in answering the specific question.", "start": 1607.045, "speaker": "SPEAKER_02"}, {"end": 1614.629, "text": "And we did that with using that feedback that we were getting.", "start": 1612.027, "speaker": "SPEAKER_02"}, {"end": 1617.77, "text": "And if you may say, well, why does that matter?", "start": 1615.609, "speaker": "SPEAKER_02"}, {"end": 1633.778, "text": "If you just take some off-the-shelf AI model that sticks things into this embedding space, that kind of puts things in the vector space that you can later find it, those are just trained on general sets of sentences, like global internet stuff.", "start": 1618.13, "speaker": "SPEAKER_02"}, {"end": 1635.419, "text": "So they'll be okay.", "start": 1634.358, "speaker": "SPEAKER_02"}, {"end": 1639.796, "text": " at finding the pieces that are relevant in my content.", "start": 1636.753, "speaker": "SPEAKER_02"}, {"end": 1641.518, "text": "But they're not going to be great.", "start": 1640.457, "speaker": "SPEAKER_02"}, {"end": 1646.723, "text": "And if you make a custom one, these scores mean nothing to you except bigger is better.", "start": 1642.599, "speaker": "SPEAKER_02"}, {"end": 1654.731, "text": "So that cross-encoder compared to just using an open AI model does a heck of a lot better.", "start": 1647.383, "speaker": "SPEAKER_02"}, {"end": 1657.053, "text": "It's much, much better.", "start": 1655.832, "speaker": "SPEAKER_02"}, {"end": 1670.781, "text": " So you can make your own little AI models that find relevant bits of content, that understand what kind of content you have, and what good content is for the specific kinds of questions that get asked on your sites.", "start": 1658.891, "speaker": "SPEAKER_02"}, {"end": 1673.764, "text": "And the whole thing becomes a virtuous loop.", "start": 1671.682, "speaker": "SPEAKER_02"}, {"end": 1678.968, "text": "So some of you may be thinking, oh my god, that's a lot of work.", "start": 1674.724, "speaker": "SPEAKER_02"}, {"end": 1680.089, "text": "Why would we do this?", "start": 1679.188, "speaker": "SPEAKER_02"}, {"end": 1682.471, "text": "Why wouldn't I just show GPT on my website?", "start": 1680.149, "speaker": "SPEAKER_02"}, {"end": 1691.439, "text": " Well, if you ask ChadGPT a question like, why do so many designs look the same?", "start": 1686.755, "speaker": "SPEAKER_02"}, {"end": 1695.622, "text": "You're going to get this generic world knowledge answer, right?", "start": 1691.459, "speaker": "SPEAKER_02"}, {"end": 1704.049, "text": "It's going to be like, well, it's because of cognitive biases, legal and regulatory constraints, economic factors, the influence of trends and cultural influences.", "start": 1695.782, "speaker": "SPEAKER_02"}, {"end": 1706.932, "text": "It just sounds like a lot of horse shit to me.", "start": 1705.511, "speaker": "SPEAKER_02"}, {"end": 1715.7, "text": " So all that work I showed you actually allows us to deliver something that mirrors the way I would actually answer a question.", "start": 1709.713, "speaker": "SPEAKER_02"}, {"end": 1718.283, "text": "Kind of like reflects my specific opinions of things.", "start": 1715.76, "speaker": "SPEAKER_02"}, {"end": 1723.83, "text": "So instead, when you ask our interface, why do you so many designs look the same?", "start": 1718.844, "speaker": "SPEAKER_02"}, {"end": 1729.937, "text": "I basically say, look, it's because designers are just looking what everybody else is doing and kind of copying it without thinking too much.", "start": 1723.89, "speaker": "SPEAKER_02"}, {"end": 1733.415, "text": " very different kind of answer.", "start": 1731.934, "speaker": "SPEAKER_02"}, {"end": 1750.548, "text": "And then the other thing you get is all these citations that I was highlighting earlier, where again, you've got a bunch of information you've published in whatever websites you have or whatever applications you have, you can reference that stuff, make sure that it's cited and allow people to go deeper into all of it.", "start": 1734.336, "speaker": "SPEAKER_02"}, {"end": 1757.578, "text": " Now, when I say go deeper, it's worth showing a little bit of example of what that actually means.", "start": 1752.133, "speaker": "SPEAKER_02"}, {"end": 1760.781, "text": "So we asked this question, why do so many designs look the same?", "start": 1757.618, "speaker": "SPEAKER_02"}, {"end": 1768.107, "text": "And that answer comes from, I mentioned a few times, videos, audio, web pages, whatever.", "start": 1762.823, "speaker": "SPEAKER_02"}, {"end": 1776.734, "text": " If you were to click on one of those videos, what we're going to actually do is inside this conversational interface, bring up that video.", "start": 1769.188, "speaker": "SPEAKER_02"}, {"end": 1778.215, "text": "I showed you an example earlier.", "start": 1776.854, "speaker": "SPEAKER_02"}, {"end": 1787.842, "text": "Take you to that part of the video where that answer is showing up inside of the video track, and then give you the ability to ask questions of it, give you a.", "start": 1778.595, "speaker": "SPEAKER_02"}, {"end": 1797.714, "text": " Summary of that video give you a full transcript that you can search against, so you can click that transcript thing and look for whatever you want inside that video file.", "start": 1789.449, "speaker": "SPEAKER_02"}, {"end": 1810.181, "text": "If it's an audio file, same thing, we'll just embed an audio player, give you again that transcript and summary, allow you to ask questions of these multi-hour long audio files right then and there.", "start": 1798.895, "speaker": "SPEAKER_02"}, {"end": 1814.664, "text": "If it is an article, we'll put the article in there.", "start": 1811.582, "speaker": "SPEAKER_02"}, {"end": 1818.564, "text": " If it's a PDF, same thing, you kind of get it.", "start": 1816.303, "speaker": "SPEAKER_02"}, {"end": 1826.628, "text": "And so again, this is an example of how this interface, this conversational interface, has basically eaten all the stuff that previously made a website.", "start": 1819.004, "speaker": "SPEAKER_02"}, {"end": 1831.17, "text": "Because now all these things appear in the conversation.", "start": 1827.689, "speaker": "SPEAKER_02"}, {"end": 1831.971, "text": "You ask a question.", "start": 1831.19, "speaker": "SPEAKER_02"}, {"end": 1833.231, "text": "Here's the video that answers it.", "start": 1832.011, "speaker": "SPEAKER_02"}, {"end": 1834.512, "text": "Now you're watching that video.", "start": 1833.291, "speaker": "SPEAKER_02"}, {"end": 1835.853, "text": "You ask that a question.", "start": 1834.972, "speaker": "SPEAKER_02"}, {"end": 1836.733, "text": "Here's an audio file.", "start": 1835.913, "speaker": "SPEAKER_02"}, {"end": 1840.395, "text": "Now you can be inside that audio file or inside that article or through a PDF.", "start": 1836.773, "speaker": "SPEAKER_02"}, {"end": 1851.405, "text": " Now I said the word PDF and my body sort of cringed a little bit, because PDFs are special, as in very painfully special.", "start": 1842.182, "speaker": "SPEAKER_02"}, {"end": 1864.03, "text": "When I was building this interface, we first started with blog articles, and then we added video, and then we added audio, and then we added PDF.", "start": 1854.626, "speaker": "SPEAKER_02"}, {"end": 1872.041, "text": " And when we added something like 400-ish PDFs to here, you could see something changed pretty dramatically.", "start": 1865.174, "speaker": "SPEAKER_02"}, {"end": 1874.564, "text": "So before the question you could ask, what is design?", "start": 1872.081, "speaker": "SPEAKER_02"}, {"end": 1877.647, "text": "It would say something like, design is the art of gradually applying constraints.", "start": 1874.584, "speaker": "SPEAKER_02"}, {"end": 1879.75, "text": "So I kind of like this answer over here.", "start": 1877.948, "speaker": "SPEAKER_02"}, {"end": 1884.595, "text": "And then I added the 400 PDFs, and all of a sudden,", "start": 1880.49, "speaker": "SPEAKER_02"}, {"end": 1892.687, "text": " This one PDF file from 2006 about dogma free design like just dominated the results.", "start": 1886.405, "speaker": "SPEAKER_02"}, {"end": 1899.989, "text": "And now it says it is directed towards human beings with the aim of solving design is the fundamental soul of human creation.", "start": 1893.327, "speaker": "SPEAKER_02"}, {"end": 1901.009, "text": "I'm like, what is this?", "start": 1900.009, "speaker": "SPEAKER_02"}, {"end": 1902.57, "text": "This doesn't sound like me.", "start": 1901.029, "speaker": "SPEAKER_02"}, {"end": 1913.962, "text": " Oh, what turned out is inside this PDF, there was like four screens of quotes from other designers about what is design.", "start": 1905.211, "speaker": "SPEAKER_02"}, {"end": 1919.889, "text": "And so the model, the information retrieval model is like, hell yeah, look at this baby, right?", "start": 1914.803, "speaker": "SPEAKER_02"}, {"end": 1920.29, "text": "And it's like.", "start": 1919.929, "speaker": "SPEAKER_02"}, {"end": 1927.913, "text": " And so the answer just becomes like this regurgitation of a whole bunch of quotes about what is design.", "start": 1922.27, "speaker": "SPEAKER_02"}, {"end": 1930.314, "text": "So that's not good, right?", "start": 1929.073, "speaker": "SPEAKER_02"}, {"end": 1939.118, "text": "And the other thing that was happening was a lot of junk is kind of showing up as part of this content.", "start": 1931.534, "speaker": "SPEAKER_02"}, {"end": 1943.399, "text": " So if you break down what's inside, a PDF is like markup language, right?", "start": 1939.838, "speaker": "SPEAKER_02"}, {"end": 1947.921, "text": "It's got so much stuff in there about how things should look as opposed to what the content is.", "start": 1943.439, "speaker": "SPEAKER_02"}, {"end": 1951.742, "text": "And so you just get pieces of content that are just crap.", "start": 1948.741, "speaker": "SPEAKER_02"}, {"end": 1955.223, "text": "I mean, look at some of these things.", "start": 1951.822, "speaker": "SPEAKER_02"}, {"end": 1957.684, "text": "1,268, the content is zero nature.", "start": 1955.243, "speaker": "SPEAKER_02"}, {"end": 1965.167, "text": "One, two, six, five, biography photo, quote, auto video in the press.", "start": 1961.145, "speaker": "SPEAKER_02"}, {"end": 1971.349, "text": " So you're like, dear AI model, please answer the question with this.", "start": 1968.547, "speaker": "SPEAKER_02"}, {"end": 1975.192, "text": "And surprising the thing works at all.", "start": 1971.67, "speaker": "SPEAKER_02"}, {"end": 1977.674, "text": "It's actually a testament to how amazing these things that it works at all.", "start": 1975.212, "speaker": "SPEAKER_02"}, {"end": 1988.083, "text": "So what we ended up doing is cleaning this up again, getting rid of a bunch of stuff that's crap and clogging up our answers, introducing a mechanism where we diversify the answer.", "start": 1978.055, "speaker": "SPEAKER_02"}, {"end": 1992.426, "text": "So like, yo, yo, yo, you can't just use this one PDF even though it loves to have design quotes.", "start": 1988.123, "speaker": "SPEAKER_02"}, {"end": 1993.828, "text": "Break it up a little bit.", "start": 1992.887, "speaker": "SPEAKER_02"}, {"end": 1999.35, "text": " And when we did that, we actually got to this really nice balance answer for what is design.", "start": 1994.568, "speaker": "SPEAKER_02"}, {"end": 2004.191, "text": "There's a PDF there, there's a video, there's two podcasts, there's a text file.", "start": 2000.11, "speaker": "SPEAKER_02"}, {"end": 2009.513, "text": "And now it's starting to really represent the diversity of content and clean it up.", "start": 2005.772, "speaker": "SPEAKER_02"}, {"end": 2017.047, "text": " So the answer got a heck of a lot better through cleaning up all this content and balancing what shows up when.", "start": 2011.823, "speaker": "SPEAKER_02"}, {"end": 2019.85, "text": "And hopefully I'm getting through to you.", "start": 2018.288, "speaker": "SPEAKER_02"}, {"end": 2022.392, "text": "You know, people ask, like, what chatbot did you use for this?", "start": 2019.87, "speaker": "SPEAKER_02"}, {"end": 2023.793, "text": "What LLM is this?", "start": 2022.852, "speaker": "SPEAKER_02"}, {"end": 2024.874, "text": "Is this Chad GPT?", "start": 2023.893, "speaker": "SPEAKER_02"}, {"end": 2026.155, "text": "Who cares what it is?", "start": 2025.234, "speaker": "SPEAKER_02"}, {"end": 2029.718, "text": "If you feed it crap, it's going to respond with crap, right?", "start": 2026.695, "speaker": "SPEAKER_02"}, {"end": 2038.224, "text": " So you really need to take a lot of the document prep and the data pipelines and the cleanup and all this stuff super seriously.", "start": 2030.578, "speaker": "SPEAKER_02"}, {"end": 2042.648, "text": "We're dealing in a world now where people are like, well, what am I going to do as a designer?", "start": 2038.705, "speaker": "SPEAKER_02"}, {"end": 2043.688, "text": "You're going to do data cleanup.", "start": 2042.668, "speaker": "SPEAKER_02"}, {"end": 2051.174, "text": "You laugh, but it's like, you know, content strategy, anybody remember that?", "start": 2046.791, "speaker": "SPEAKER_02"}, {"end": 2054.117, "text": "That was like a hot button, I don't know, 20 years ago or something like that.", "start": 2051.214, "speaker": "SPEAKER_02"}, {"end": 2059.928, "text": " It's a big part of it, and this becomes more like direction, right?", "start": 2056.727, "speaker": "SPEAKER_02"}, {"end": 2064.789, "text": "Art direction, content direction, model direction, and steering.", "start": 2060.908, "speaker": "SPEAKER_02"}, {"end": 2069.891, "text": "Somebody needs to have an opinion when I was showing you those two answers on what is design.", "start": 2065.35, "speaker": "SPEAKER_02"}, {"end": 2074.392, "text": "Somebody needs to have an opinion on what is the answer that you want to deliver with the system.", "start": 2069.911, "speaker": "SPEAKER_02"}, {"end": 2075.092, "text": "What should it be?", "start": 2074.452, "speaker": "SPEAKER_02"}, {"end": 2076.793, "text": "What can't it be?", "start": 2075.933, "speaker": "SPEAKER_02"}, {"end": 2078.753, "text": "What is not okay?", "start": 2077.833, "speaker": "SPEAKER_02"}, {"end": 2080.614, "text": "What is okay?", "start": 2080.034, "speaker": "SPEAKER_02"}, {"end": 2095.652, "text": " Now, we could do even better, because recent advances, and we did this about a month or two ago, in vision models have made it possible for us to totally change the game and what we can do from images and PDFs.", "start": 2083.365, "speaker": "SPEAKER_02"}, {"end": 2100.334, "text": "So before, we spent a bunch of time trying to get models to do text extraction.", "start": 2096.372, "speaker": "SPEAKER_02"}, {"end": 2103.796, "text": "We got pretty good at extracting all the real content from a file.", "start": 2100.594, "speaker": "SPEAKER_02"}, {"end": 2111.2, "text": "But now what we can do is actually generate things that say something like, the chart illustrates the impact of A-B testing results from 2012 to 2019.", "start": 2104.316, "speaker": "SPEAKER_02"}, {"end": 2121.824, "text": " The gray line represents this, suggesting a discrepancy between the positive outcomes indicated by A-B testing and actual business results achieved.", "start": 2113.021, "speaker": "SPEAKER_02"}, {"end": 2125.465, "text": "What do you think is going to make your responses better?", "start": 2123.105, "speaker": "SPEAKER_02"}, {"end": 2127.326, "text": "This one.", "start": 2126.966, "speaker": "SPEAKER_02"}, {"end": 2129.547, "text": "Here's another example.", "start": 2128.687, "speaker": "SPEAKER_02"}, {"end": 2132.468, "text": "Here's a screenshot from my Mind the Gap talk.", "start": 2129.667, "speaker": "SPEAKER_02"}, {"end": 2134.028, "text": "This is what we had before.", "start": 2132.928, "speaker": "SPEAKER_02"}, {"end": 2138.07, "text": "Good job getting all that content and image out of there.", "start": 2135.229, "speaker": "SPEAKER_02"}, {"end": 2139.77, "text": "That's a pretty tough task.", "start": 2138.27, "speaker": "SPEAKER_02"}, {"end": 2140.731, "text": "Did a good job.", "start": 2140.131, "speaker": "SPEAKER_02"}, {"end": 2143.016, "text": " This is so much better.", "start": 2141.755, "speaker": "SPEAKER_02"}, {"end": 2154.205, "text": "And so now what we do is we have a pipeline that creates this kind of content from every single screen in the PDF and from every single image.", "start": 2146.519, "speaker": "SPEAKER_02"}, {"end": 2165.634, "text": "And when you then apply that to your embedding index and you change the content again, you once more have a very different set of answers for what is design.", "start": 2155.386, "speaker": "SPEAKER_02"}, {"end": 2167.996, "text": "And in this case, I think it's getting better.", "start": 2165.654, "speaker": "SPEAKER_02"}, {"end": 2170.678, "text": "Like every time I've done this, this answer has gotten better.", "start": 2168.016, "speaker": "SPEAKER_02"}, {"end": 2181.082, "text": " Every time we've enriched the content and spent the time to make sure that what we're feeding in and how we're processing it is a lot better.", "start": 2171.959, "speaker": "SPEAKER_02"}, {"end": 2183.703, "text": "So this is a process of constant adjustment.", "start": 2181.462, "speaker": "SPEAKER_02"}, {"end": 2187.084, "text": "It's the same thing like people say, OK, I launched the website.", "start": 2184.203, "speaker": "SPEAKER_02"}, {"end": 2187.464, "text": "I'm done.", "start": 2187.104, "speaker": "SPEAKER_02"}, {"end": 2188.164, "text": "No, you're not.", "start": 2187.664, "speaker": "SPEAKER_02"}, {"end": 2189.465, "text": "That's the beginning of it.", "start": 2188.504, "speaker": "SPEAKER_02"}, {"end": 2191.405, "text": "Oh, I added a chat bot.", "start": 2190.345, "speaker": "SPEAKER_02"}, {"end": 2191.786, "text": "I'm done.", "start": 2191.425, "speaker": "SPEAKER_02"}, {"end": 2193.046, "text": "No, that's the beginning of it.", "start": 2191.826, "speaker": "SPEAKER_02"}, {"end": 2208.633, "text": " So for PDFs, we need lots of cleaning, but now you can see that there's these, in addition to some of the text-based tasks, we've actually got some vision tasks to make sense of all that information and make it useful for these models.", "start": 2195.182, "speaker": "SPEAKER_02"}, {"end": 2220.303, "text": "The other really cool thing that this enables is once you start processing images and understanding semantically what's in an image, you can actually start to respond with images and get multimodal.", "start": 2209.354, "speaker": "SPEAKER_02"}, {"end": 2223.766, "text": " So now your website doesn't have to just like spit out text.", "start": 2221.044, "speaker": "SPEAKER_02"}, {"end": 2225.007, "text": "It can actually do things like this.", "start": 2223.906, "speaker": "SPEAKER_02"}, {"end": 2228.69, "text": "So I ask a question, what's the difference between AR and VR?", "start": 2225.508, "speaker": "SPEAKER_02"}, {"end": 2232.733, "text": "And like literally, this image shows the spectrum from VR to AR.", "start": 2229.191, "speaker": "SPEAKER_02"}, {"end": 2242.501, "text": "You can see that it creates a completely digital world while AR, it's not only putting the image in the right place in the response, but it's actually referencing it in its response.", "start": 2232.814, "speaker": "SPEAKER_02"}, {"end": 2246.745, "text": "So this creates a whole new set of capabilities.", "start": 2244.283, "speaker": "SPEAKER_02"}, {"end": 2253.542, "text": " Now you can start to respond with pages of PDFs, too, in the same way.", "start": 2248.52, "speaker": "SPEAKER_02"}, {"end": 2260.925, "text": "It knows what's in that page of the PDF because we've spent all this time to process it and put it in the embedding index.", "start": 2253.602, "speaker": "SPEAKER_02"}, {"end": 2268.568, "text": "We've trained a retriever to find it when it's most relevant, and then we can start to assemble answers that make a ton more sense.", "start": 2261.425, "speaker": "SPEAKER_02"}, {"end": 2271.669, "text": "Okay, that's a lot of stuff.", "start": 2270.388, "speaker": "SPEAKER_02"}, {"end": 2276.353, "text": " But hopefully, big picture, let's take a step back.", "start": 2273.612, "speaker": "SPEAKER_02"}, {"end": 2278.094, "text": "You have website.", "start": 2277.054, "speaker": "SPEAKER_02"}, {"end": 2280.475, "text": "Website have stuff in it.", "start": 2279.074, "speaker": "SPEAKER_02"}, {"end": 2281.636, "text": "Yes?", "start": 2281.395, "speaker": "SPEAKER_02"}, {"end": 2285.317, "text": "Okay, you all know website have stuff.", "start": 2282.876, "speaker": "SPEAKER_02"}, {"end": 2287.418, "text": "Different kind of stuff.", "start": 2286.338, "speaker": "SPEAKER_02"}, {"end": 2291.7, "text": "AI models on stuff.", "start": 2288.699, "speaker": "SPEAKER_02"}, {"end": 2294.501, "text": "Very cool.", "start": 2293.961, "speaker": "SPEAKER_02"}, {"end": 2298.063, "text": "Whole new ball game.", "start": 2297.163, "speaker": "SPEAKER_02"}, {"end": 2304.195, "text": " And I think we're just scratching at the surface of this.", "start": 2301.914, "speaker": "SPEAKER_02"}, {"end": 2312.2, "text": "By the way, this is why I have these little AI guys, because they're hustling through all these little AI models, trying to figure out what to do and how to use what thing, when, and how.", "start": 2304.235, "speaker": "SPEAKER_02"}, {"end": 2318.323, "text": "Behind the scenes, they're using those language models, those vision models, everything.", "start": 2314.281, "speaker": "SPEAKER_02"}, {"end": 2324.506, "text": "And so the end result here is not some... Hold on.", "start": 2319.123, "speaker": "SPEAKER_02"}, {"end": 2326.928, "text": "And the end result here is not some...", "start": 2325.387, "speaker": "SPEAKER_02"}, {"end": 2338.317, "text": " Scary AI thing, but instead it's a new way to use all this content, right?", "start": 2333.214, "speaker": "SPEAKER_02"}, {"end": 2342.219, "text": "To kind of mirror real world experiences.", "start": 2338.817, "speaker": "SPEAKER_02"}, {"end": 2352.484, "text": "You don't have to go through those 10 blue links and click on them and then find the part of the article that kind of relates to the thing that you're trying to go browse through long lists of things.", "start": 2342.239, "speaker": "SPEAKER_02"}, {"end": 2356.686, "text": "You can just say, what you're asking, right?", "start": 2352.504, "speaker": "SPEAKER_02"}, {"end": 2361.309, "text": "And so you could say something like, hey, what should designers do about AI?", "start": 2357.687, "speaker": "SPEAKER_02"}, {"end": 2365.12, "text": " Yeah, it's going to reference some talks.", "start": 2363.479, "speaker": "SPEAKER_02"}, {"end": 2367.082, "text": "It's going to give you an answer.", "start": 2365.24, "speaker": "SPEAKER_02"}, {"end": 2368.823, "text": "And away you go.", "start": 2368.283, "speaker": "SPEAKER_02"}, {"end": 2376.43, "text": "So these conversational interfaces, whatever we want to call them, or whatever they become in the future.", "start": 2370.625, "speaker": "SPEAKER_02"}, {"end": 2378.792, "text": "I launched this service on my website.", "start": 2377.11, "speaker": "SPEAKER_02"}, {"end": 2380.493, "text": "By the way, you can all, I should have said this again.", "start": 2378.812, "speaker": "SPEAKER_02"}, {"end": 2381.494, "text": "You can all go play with it.", "start": 2380.513, "speaker": "SPEAKER_02"}, {"end": 2383.616, "text": "Ask.LukeW.com.", "start": 2381.674, "speaker": "SPEAKER_02"}, {"end": 2385.377, "text": "Go ahead, ask whatever questions you want.", "start": 2383.656, "speaker": "SPEAKER_02"}, {"end": 2386.338, "text": "Don't crash it.", "start": 2385.697, "speaker": "SPEAKER_02"}, {"end": 2393.495, "text": " But we've been doing this for almost a year and a half now.", "start": 2389.352, "speaker": "SPEAKER_02"}, {"end": 2398.559, "text": "And like every couple months, there's some big interesting change where we can do new things.", "start": 2393.715, "speaker": "SPEAKER_02"}, {"end": 2400.62, "text": "Now we're doing these inline citations.", "start": 2398.819, "speaker": "SPEAKER_02"}, {"end": 2402.241, "text": "Now we're putting images in there.", "start": 2400.68, "speaker": "SPEAKER_02"}, {"end": 2405.023, "text": "And the models are actually referencing the images.", "start": 2402.702, "speaker": "SPEAKER_02"}, {"end": 2410.027, "text": "Next up, we're going to try doing that within inline video files.", "start": 2405.624, "speaker": "SPEAKER_02"}, {"end": 2411.088, "text": "Who knows?", "start": 2410.747, "speaker": "SPEAKER_02"}, {"end": 2411.988, "text": "Lots of stuff to do.", "start": 2411.248, "speaker": "SPEAKER_02"}, {"end": 2418.033, "text": "So on behalf of me and all my little AI guys,", "start": 2413.029, "speaker": "SPEAKER_02"}, {"end": 2421.723, "text": " Thank you.", "start": 2421.343, "speaker": "SPEAKER_02"}, {"end": 2433.629, "text": "And I have one question for you.", "start": 2432.369, "speaker": "SPEAKER_02"}, {"end": 2436.571, "text": "Are we going to actually find out what the bananas are for?", "start": 2434.17, "speaker": "SPEAKER_02"}, {"end": 2441.093, "text": "Because they've been sitting there all day, and I'm just dying to know.", "start": 2436.631, "speaker": "SPEAKER_02"}, {"end": 2442.694, "text": "OK.", "start": 2442.374, "speaker": "SPEAKER_02"}, {"end": 2444.175, "text": "No, no, no.", "start": 2443.775, "speaker": "SPEAKER_02"}, {"end": 2444.495, "text": "Thank you.", "start": 2444.195, "speaker": "SPEAKER_02"}, {"end": 2444.935, "text": "That's fine.", "start": 2444.515, "speaker": "SPEAKER_02"}, {"end": 2445.856, "text": "All right.", "start": 2445.516, "speaker": "SPEAKER_00"}, {"end": 2447.837, "text": "They're still here.", "start": 2447.157, "speaker": "SPEAKER_02"}, {"end": 2448.737, "text": "That's right.", "start": 2448.037, "speaker": "SPEAKER_00"}, {"end": 2449.658, "text": "Toby, you want a banana?", "start": 2448.877, "speaker": "SPEAKER_00"}, {"end": 2454.111, "text": " I mean, I'm not going to throw bananas at you.", "start": 2452.07, "speaker": "SPEAKER_00"}, {"end": 2456.132, "text": "That's a bit disrespectful, I think.", "start": 2454.151, "speaker": "SPEAKER_00"}, {"end": 2459.713, "text": "Thank you so much, Luke, for the insights.", "start": 2456.992, "speaker": "SPEAKER_00"}, {"end": 2461.254, "text": "Can we switch?", "start": 2460.674, "speaker": "SPEAKER_00"}, {"end": 2461.934, "text": "Oh, sure.", "start": 2461.614, "speaker": "SPEAKER_00"}, {"end": 2463.395, "text": "Am I in your chair?", "start": 2462.194, "speaker": "SPEAKER_02"}, {"end": 2466.976, "text": "No, I just feel... I have to sit near the bananas after that comment.", "start": 2463.455, "speaker": "SPEAKER_02"}, {"end": 2468.477, "text": "Oh, this is my spot.", "start": 2467.516, "speaker": "SPEAKER_00"}, {"end": 2474.852, "text": " Oh, they blend in, that's okay.", "start": 2473.311, "speaker": "SPEAKER_00"}, {"end": 2486.562, "text": "So I'm wondering at this point, actually when you think about like a selection of a model, I mean you basically said that it doesn't matter as long as you have like the proper information architecture, content strategy in place and all.", "start": 2475.173, "speaker": "SPEAKER_00"}, {"end": 2491.647, "text": "So you would say that there is basically no preference in terms of what you're going for?", "start": 2486.983, "speaker": "SPEAKER_00"}, {"end": 2509.865, "text": " It depends on what you're trying to do so if you want a general high-level Open-ended task a very large foundational model Claude GPT those things is going to be off the shelf what you want because it's post-trained if you want to do a very specific task", "start": 2492.808, "speaker": "SPEAKER_02"}, {"end": 2521.031, "text": " That benefit of being very broadly trained to do anything is actually a negative because they're not specifically trained to do one particular thing a very particular way.", "start": 2510.726, "speaker": "SPEAKER_02"}, {"end": 2521.411, "text": "Right.", "start": 2521.231, "speaker": "SPEAKER_00"}, {"end": 2530.596, "text": "So does it mean that then it's in some specific cases like an enterprise world where you want to kind of really optimize a specific task instead of going for large language models to go for small language models?", "start": 2521.431, "speaker": "SPEAKER_00"}, {"end": 2534.24, "text": " The architecture is usually a combination of multiple things.", "start": 2531.176, "speaker": "SPEAKER_02"}, {"end": 2539.627, "text": "So you use smaller fine-tuned models for very specific tasks where you want to do something concrete.", "start": 2534.941, "speaker": "SPEAKER_02"}, {"end": 2541.79, "text": "Let's talk about Brad's example earlier.", "start": 2539.687, "speaker": "SPEAKER_02"}, {"end": 2545.995, "text": "If I want my particular pattern library,", "start": 2542.291, "speaker": "SPEAKER_02"}, {"end": 2559.029, "text": " to be a very strong influence on the creation of components, then I'm going to create a bunch of training examples and I'm going to fine tune a model that's already good at writing code to write exactly my specific kind of code.", "start": 2546.616, "speaker": "SPEAKER_02"}, {"end": 2567.478, "text": "But in that same pipeline, I may use something bigger to answer like very high level things like make a plan for what this web page should be.", "start": 2559.429, "speaker": "SPEAKER_02"}, {"end": 2579.644, "text": " Right, so that is a very broad high-level world knowledge task and then like turn it into this code using this library is a very different kind of specific task I see use a smaller fine-tune model, right?", "start": 2568.298, "speaker": "SPEAKER_02"}, {"end": 2583.986, "text": "I'm also wondering just how come like actually those things like citations, right?", "start": 2579.884, "speaker": "SPEAKER_00"}, {"end": 2595.633, "text": "And and some of those nice these features that you find that in perplexity and in Claude like just sharing the actual output being able to Reef kind of fork it like what you get into this is fantastic, but most", "start": 2584.087, "speaker": "SPEAKER_00"}, {"end": 2598.034, "text": " AI experiences are not like that.", "start": 2596.613, "speaker": "SPEAKER_00"}, {"end": 2603.957, "text": "So I'm actually wondering, I mean, maybe that's my personal experience, but I just want to share and I'm wondering what you think as well.", "start": 2598.094, "speaker": "SPEAKER_00"}, {"end": 2609.24, "text": "I think over the last maybe three weeks, I have used Google once.", "start": 2604.717, "speaker": "SPEAKER_00"}, {"end": 2611.881, "text": "Maybe just once.", "start": 2611.101, "speaker": "SPEAKER_00"}, {"end": 2614.142, "text": "I mean, Google Maps is a different story, right?", "start": 2612.441, "speaker": "SPEAKER_00"}, {"end": 2617.484, "text": "But I mean, like, like literally searching and going through links, really rare.", "start": 2614.162, "speaker": "SPEAKER_00"}, {"end": 2622.427, "text": "But at the same time, over the last two months, I maybe used CHGPT once.", "start": 2617.944, "speaker": "SPEAKER_00"}, {"end": 2627.661, "text": " So the only thing I'm paying for is perplexity, and I'm using it all the time.", "start": 2623.84, "speaker": "SPEAKER_00"}, {"end": 2629.641, "text": "And I mean, like, all the time.", "start": 2627.961, "speaker": "SPEAKER_00"}, {"end": 2630.942, "text": "And I get the reference.", "start": 2629.681, "speaker": "SPEAKER_00"}, {"end": 2633.702, "text": "Perplexity employee.", "start": 2632.582, "speaker": "SPEAKER_00"}, {"end": 2634.782, "text": "Perplexity?", "start": 2633.982, "speaker": "SPEAKER_00"}, {"end": 2635.402, "text": "No, no.", "start": 2634.862, "speaker": "SPEAKER_00"}, {"end": 2637.143, "text": "Perplexity stock owner.", "start": 2635.923, "speaker": "SPEAKER_00"}, {"end": 2645.745, "text": "But I think for me, the reason why I'm saying that is that, to me, perplexity is doing this incredible job and kind of combining all these different patterns that you were also mentioning.", "start": 2637.263, "speaker": "SPEAKER_00"}, {"end": 2648.125, "text": "But why is it not more like a norm?", "start": 2646.425, "speaker": "SPEAKER_00"}, {"end": 2649.966, "text": "Why is the norm still just chatbot?", "start": 2648.145, "speaker": "SPEAKER_00"}, {"end": 2651.646, "text": "Why are most web pages pieces of shit?", "start": 2650.006, "speaker": "SPEAKER_00"}, {"end": 2662.552, "text": " Yeah, but, well, if you look at your GPT, right, it's, you know, over two years, it feels like it's better, but not better, if it makes sense.", "start": 2652.206, "speaker": "SPEAKER_00"}, {"end": 2666.814, "text": "My joke was not a joke, really, right?", "start": 2664.253, "speaker": "SPEAKER_02"}, {"end": 2675.178, "text": "Like, if you look at the early stages of mobile apps, the vast majority of mobile apps at the beginning were moving web pages into a tiny little screen.", "start": 2666.874, "speaker": "SPEAKER_02"}, {"end": 2675.839, "text": "They were pretty bad.", "start": 2675.218, "speaker": "SPEAKER_02"}, {"end": 2691.893, "text": " They're trying to apply very prior patterns to a new medium, and it just takes a while to learn how to use that medium, what the experiences should be, and there's going to be more evolution beyond what I showed and what perplexity is doing, whatever.", "start": 2676.739, "speaker": "SPEAKER_02"}, {"end": 2694.176, "text": "We're still super early in this stuff.", "start": 2692.334, "speaker": "SPEAKER_02"}, {"end": 2699.677, "text": " But at the same time, I feel like there is a bit, I mean, we had a very intense discussion about that yesterday.", "start": 2694.716, "speaker": "SPEAKER_00"}, {"end": 2703.918, "text": "Because again, when you look at how AI is perceived outside of the US, it's very different.", "start": 2700.157, "speaker": "SPEAKER_00"}, {"end": 2708.159, "text": "Like in Europe, whenever you start having conversations, there was a conference around you at AI.", "start": 2703.958, "speaker": "SPEAKER_00"}, {"end": 2714.321, "text": "And then there is this sense of frustration, I guess, which would be the right word.", "start": 2708.699, "speaker": "SPEAKER_00"}, {"end": 2720.562, "text": "And the reason for that is very simple, because many people are speaking about, you know, 2022 AI comes up, everybody's like, wow, that's magical.", "start": 2714.341, "speaker": "SPEAKER_00"}, {"end": 2732.168, "text": " Now, everybody's like, when people speak about AI, it's like it feels fake or hallucinating or slow because you always have to wait for your GPT out.", "start": 2721.542, "speaker": "SPEAKER_00"}, {"end": 2733.669, "text": "You can't render hands.", "start": 2732.188, "speaker": "SPEAKER_02"}, {"end": 2734.249, "text": "What's wrong?", "start": 2733.729, "speaker": "SPEAKER_02"}, {"end": 2735.37, "text": "Yeah, like all those things.", "start": 2734.309, "speaker": "SPEAKER_00"}, {"end": 2739.032, "text": "But I mean, I'm just wondering, where are we now with the idea?", "start": 2735.63, "speaker": "SPEAKER_00"}, {"end": 2741.553, "text": "Do we feel like it's like really groundbreaking innovation?", "start": 2739.052, "speaker": "SPEAKER_00"}, {"end": 2746.516, "text": "Because everybody was expecting, it felt like a big breakthrough in health care and whatnot.", "start": 2741.593, "speaker": "SPEAKER_00"}, {"end": 2749.218, "text": " Is it where we are?", "start": 2748.297, "speaker": "SPEAKER_00"}, {"end": 2750.118, "text": "Or is it not?", "start": 2749.278, "speaker": "SPEAKER_00"}, {"end": 2751.759, "text": "What is the state of it now?", "start": 2750.158, "speaker": "SPEAKER_00"}, {"end": 2755.1, "text": "I mean, I literally showed you a digital version of my brain.", "start": 2751.979, "speaker": "SPEAKER_02"}, {"end": 2759.402, "text": "Which is fantastic to see, but you also have it all documented neatly, I think.", "start": 2755.601, "speaker": "SPEAKER_00"}, {"end": 2764.145, "text": "We should have digital versions of your brain and everybody else's brains.", "start": 2760.223, "speaker": "SPEAKER_02"}, {"end": 2770.003, "text": " And then you have this... Then I'll know what you're doing with these.", "start": 2767.682, "speaker": "SPEAKER_00"}, {"end": 2771.744, "text": "Oh, yeah, yeah, yeah.", "start": 2770.784, "speaker": "SPEAKER_02"}, {"end": 2772.485, "text": "That's right.", "start": 2772.064, "speaker": "SPEAKER_02"}, {"end": 2772.865, "text": "That's right.", "start": 2772.505, "speaker": "SPEAKER_02"}, {"end": 2775.206, "text": "But do you see... No, it's the same answer.", "start": 2773.125, "speaker": "SPEAKER_02"}, {"end": 2776.307, "text": "We're still so early.", "start": 2775.266, "speaker": "SPEAKER_02"}, {"end": 2776.867, "text": "Yeah.", "start": 2776.687, "speaker": "SPEAKER_01"}, {"end": 2777.828, "text": "We're going to figure this out.", "start": 2776.947, "speaker": "SPEAKER_02"}, {"end": 2784.231, "text": "Like, when the iPhone came out in 2007, was there a TikTok?", "start": 2777.868, "speaker": "SPEAKER_02"}, {"end": 2786.653, "text": "Nope.", "start": 2786.452, "speaker": "SPEAKER_02"}, {"end": 2790.715, "text": "It's like everybody's brain now fused to TikTok in a terrible way.", "start": 2787.153, "speaker": "SPEAKER_02"}, {"end": 2790.895, "text": "Yeah.", "start": 2790.775, "speaker": "SPEAKER_02"}, {"end": 2793.019, "text": " Right.", "start": 2792.739, "speaker": "SPEAKER_00"}, {"end": 2807.655, "text": "So like those, it's just, you know, there's a ton of time which will put, so if you ask me, I think as I tried to illustrate there, you can basically put AI throughout your whole stack.", "start": 2793.079, "speaker": "SPEAKER_02"}, {"end": 2823.209, "text": " From the user experience all the way down to how you store information right to like what your data structures are or no data structures whatsoever your data structures are just text and then you can basically build up and down from there like that's.", "start": 2808.476, "speaker": "SPEAKER_02"}, {"end": 2824.75, "text": " Pretty crazy.", "start": 2823.869, "speaker": "SPEAKER_02"}, {"end": 2835.44, "text": "Even in all these other previous kind of like major technology changes we had, we didn't have anything that is the entire stack all at once.", "start": 2825.13, "speaker": "SPEAKER_02"}, {"end": 2840.004, "text": "Parts of the stack changed and that had massive repercussions on the industry.", "start": 2835.66, "speaker": "SPEAKER_02"}, {"end": 2844.889, "text": "Web, mobile, you name it, cloud, more on the lower side of the stack.", "start": 2840.224, "speaker": "SPEAKER_02"}, {"end": 2846.67, "text": "But now the whole stack", "start": 2845.289, "speaker": "SPEAKER_02"}, {"end": 2851.078, "text": " is basically being rethought and changed.", "start": 2847.771, "speaker": "SPEAKER_02"}, {"end": 2857.23, "text": "So I think the changes that we're gonna see of this, again, personally, I believe are gonna be more dramatic than those.", "start": 2851.539, "speaker": "SPEAKER_02"}, {"end": 2862.925, "text": " Actually, I should probably also cover some questions from the audience, by the way.", "start": 2859.584, "speaker": "SPEAKER_00"}, {"end": 2864.586, "text": "We can talk about your questions.", "start": 2863.406, "speaker": "SPEAKER_00"}, {"end": 2866.346, "text": "No, no, that's okay.", "start": 2865.046, "speaker": "SPEAKER_00"}, {"end": 2868.527, "text": "We can have a conversation anyway.", "start": 2867.407, "speaker": "SPEAKER_00"}, {"end": 2873.208, "text": "There was a question that came up quickly.", "start": 2870.308, "speaker": "SPEAKER_00"}, {"end": 2875.509, "text": "Can you talk about content strategy for AI?", "start": 2873.248, "speaker": "SPEAKER_00"}, {"end": 2876.449, "text": "What does it mean?", "start": 2875.649, "speaker": "SPEAKER_00"}, {"end": 2882.091, "text": "How do you maintain your application model for not just existing, but, ooh, that just jumped.", "start": 2876.809, "speaker": "SPEAKER_00"}, {"end": 2884.131, "text": "But new content, yeah.", "start": 2883.171, "speaker": "SPEAKER_00"}, {"end": 2885.932, "text": "New content and something on that.", "start": 2884.151, "speaker": "SPEAKER_00"}, {"end": 2887.273, "text": " Fascinating topic.", "start": 2886.292, "speaker": "SPEAKER_02"}, {"end": 2889.274, "text": "I don't think anybody really knows yet.", "start": 2887.833, "speaker": "SPEAKER_02"}, {"end": 2903.761, "text": "The one thing I didn't show with the Ask Luke system that I can, if people are interested, is when questions come in that I don't have answers to inside my content corpus, but I actually have an answer in my head.", "start": 2890.194, "speaker": "SPEAKER_02"}, {"end": 2908.423, "text": "I've gotten into this workflow where I built an admin system where I just press a button and talk.", "start": 2904.341, "speaker": "SPEAKER_02"}, {"end": 2918.002, "text": " It records all that, like I don't have to write a blog article or anything like that, it just slurps it all up and that becomes part of the content corpus and it grows.", "start": 2909.798, "speaker": "SPEAKER_02"}, {"end": 2931.349, "text": "And so behind the scenes over the past year and a half, I've generated something like 500 equivalent articles, which imagine publishing 500 blog articles in a year.", "start": 2918.883, "speaker": "SPEAKER_02"}, {"end": 2933.23, "text": "You'd be crazy, right?", "start": 2931.929, "speaker": "SPEAKER_02"}, {"end": 2933.85, "text": "It doesn't happen.", "start": 2933.25, "speaker": "SPEAKER_02"}, {"end": 2934.351, "text": "But now,", "start": 2933.97, "speaker": "SPEAKER_02"}, {"end": 2942.755, "text": " Question comes in, I know what people are asking about, and if I have an answer, I just hit record, talk, don't care about the format, don't care about the process.", "start": 2935.211, "speaker": "SPEAKER_02"}, {"end": 2952.059, "text": "And so that is a very different way of thinking about content strategy, because it's such a precious thing before, and it's so structured.", "start": 2943.335, "speaker": "SPEAKER_02"}, {"end": 2958.362, "text": "And now you can, again, basically use natural language as your data structure.", "start": 2953.099, "speaker": "SPEAKER_02"}, {"end": 2963.988, "text": " I've personally found it a tough bit to flip in my head.", "start": 2961.025, "speaker": "SPEAKER_02"}, {"end": 2970.737, "text": "For a long time, I was still like, well, I have to then format it into an article, and I have to remove the parts that are... No, you don't.", "start": 2964.87, "speaker": "SPEAKER_02"}, {"end": 2974.717, "text": " Drop it all in there and let the system figure it out.", "start": 2972.355, "speaker": "SPEAKER_00"}, {"end": 2979.161, "text": "And when you think about this type of data consumption, that's another question that came up and processing.", "start": 2975.098, "speaker": "SPEAKER_00"}, {"end": 2981.684, "text": "How does it impact document format structures?", "start": 2979.242, "speaker": "SPEAKER_00"}, {"end": 2984.226, "text": "Are we moving away from PDFs for good?", "start": 2982.304, "speaker": "SPEAKER_00"}, {"end": 2988.25, "text": "No, because people still want PDFs as output, which is funny.", "start": 2985.387, "speaker": "SPEAKER_02"}, {"end": 2992.334, "text": "One of the requests I have is like, can you turn that answer into a PDF?", "start": 2989.651, "speaker": "SPEAKER_02"}, {"end": 2997.527, "text": " Well, now we can turn it into audio podcast.", "start": 2995.566, "speaker": "SPEAKER_00"}, {"end": 2999.608, "text": "Yeah, turn it into a podcast.", "start": 2997.567, "speaker": "SPEAKER_02"}, {"end": 3000.869, "text": "It's funny.", "start": 3000.409, "speaker": "SPEAKER_02"}, {"end": 3011.835, "text": "I always, not always, but I have in the past made fun of Microsoft Copilot because Microsoft Copilot sits alongside Microsoft Office and it's always like, you're making a Word document.", "start": 3000.909, "speaker": "SPEAKER_02"}, {"end": 3013.436, "text": "Do you want to turn it into a PowerPoint?", "start": 3011.855, "speaker": "SPEAKER_02"}, {"end": 3015.457, "text": "And then it's like, you're making a PowerPoint.", "start": 3013.956, "speaker": "SPEAKER_02"}, {"end": 3017.058, "text": "Do you want to turn it into a Word document?", "start": 3015.477, "speaker": "SPEAKER_02"}, {"end": 3019.68, "text": " What am I doing?", "start": 3018.458, "speaker": "SPEAKER_00"}, {"end": 3022.222, "text": "Right?", "start": 3019.7, "speaker": "SPEAKER_00"}, {"end": 3025.326, "text": "Pat is wondering how do you test AI for hallucination?", "start": 3022.543, "speaker": "SPEAKER_00"}, {"end": 3029.75, "text": "We are running a financial application and we can't suggest inaccurate recommendations or insights.", "start": 3025.886, "speaker": "SPEAKER_00"}, {"end": 3030.951, "text": "Accuracy is very critical.", "start": 3029.87, "speaker": "SPEAKER_00"}, {"end": 3036.117, "text": "We haven't started working on AI yet because we're afraid of giving wrong suggestions to our users.", "start": 3031.372, "speaker": "SPEAKER_00"}, {"end": 3044.087, "text": " Yep, so the reason why I did this little Ask Luke project year and a half ago or whatever was because of that exact question.", "start": 3036.437, "speaker": "SPEAKER_02"}, {"end": 3048.393, "text": "It's very difficult to assess, is this right or wrong?", "start": 3044.608, "speaker": "SPEAKER_02"}, {"end": 3051.217, "text": "When I showed the answer there, where did Luke go to school?", "start": 3048.453, "speaker": "SPEAKER_02"}, {"end": 3052.499, "text": "I know the answer.", "start": 3051.818, "speaker": "SPEAKER_02"}, {"end": 3053.44, "text": "Do any of you?", "start": 3052.999, "speaker": "SPEAKER_02"}, {"end": 3054.521, "text": " No.", "start": 3054.361, "speaker": "SPEAKER_02"}, {"end": 3057.163, "text": "So how can you tell if the system is right or wrong?", "start": 3054.701, "speaker": "SPEAKER_02"}, {"end": 3065.629, "text": "And the reason we built that system on my corpus is because I know all that content, and I could assess and verify what is wrong or right.", "start": 3057.824, "speaker": "SPEAKER_02"}, {"end": 3070.953, "text": "So when you have that capability, then you can train the system to be like, that's the correct answer.", "start": 3066.33, "speaker": "SPEAKER_02"}, {"end": 3071.734, "text": "That's the wrong answer.", "start": 3070.973, "speaker": "SPEAKER_02"}, {"end": 3072.494, "text": "That's the correct answer.", "start": 3071.754, "speaker": "SPEAKER_02"}, {"end": 3073.255, "text": "That's the wrong answer.", "start": 3072.514, "speaker": "SPEAKER_02"}, {"end": 3075.677, "text": "I literally have an interface that", "start": 3073.675, "speaker": "SPEAKER_02"}, {"end": 3082.855, "text": " our AI researcher made for me, which is like, vote on the response, which one is correct.", "start": 3077.472, "speaker": "SPEAKER_02"}, {"end": 3088.399, "text": "And I have sat there for hours being like left, right, left, right, left, right.", "start": 3083.216, "speaker": "SPEAKER_02"}, {"end": 3091.2, "text": "But that was, you know, a long time ago.", "start": 3088.779, "speaker": "SPEAKER_02"}, {"end": 3093.622, "text": "Now I think the systems are a lot better.", "start": 3091.661, "speaker": "SPEAKER_02"}, {"end": 3098.904, "text": " At that, and it's obviously an issue that everybody's aware of in the space.", "start": 3094.322, "speaker": "SPEAKER_02"}, {"end": 3103.006, "text": "I think it's becoming less and less of a concern.", "start": 3100.725, "speaker": "SPEAKER_02"}, {"end": 3106.848, "text": "And especially if you do the type of thing that I showed there, right?", "start": 3103.686, "speaker": "SPEAKER_02"}, {"end": 3113.151, "text": "It's not foolproof, but the odds of things going off the rails are like this as opposed to...", "start": 3107.208, "speaker": "SPEAKER_02"}, {"end": 3117.713, "text": " like this and it'll get better and better and they'll be more fine tuning that you can do it.", "start": 3113.951, "speaker": "SPEAKER_02"}, {"end": 3126.197, "text": "And so how much time did it take from the very beginning when you just had all the documents to more or less decent results?", "start": 3118.053, "speaker": "SPEAKER_00"}, {"end": 3129.139, "text": "This is the thing about AI.", "start": 3126.257, "speaker": "SPEAKER_02"}, {"end": 3134.141, "text": "You can very quickly get to a compelling demo that's like 80% or 90% of what you want.", "start": 3129.599, "speaker": "SPEAKER_02"}, {"end": 3155.426, "text": " Right and we saw a bunch of that today I think in some of the earlier presentations like oh look I type this into Blank and out pops a thing That's kind of what I want the next 10% is like 90% of the work right and that's true in other Domains as well, but it's particularly true in AI to stitch together something.", "start": 3136.322, "speaker": "SPEAKER_02"}, {"end": 3157.047, "text": "That's working pretty good", "start": 3155.446, "speaker": "SPEAKER_02"}, {"end": 3159.824, "text": " Super fast, very easy.", "start": 3158.243, "speaker": "SPEAKER_02"}, {"end": 3165.426, "text": "To make something that's really robust and works for a long time, really long slog, lots of work.", "start": 3159.884, "speaker": "SPEAKER_02"}, {"end": 3170.509, "text": "Maybe final question, then of course we'll have more conversations at the party, I'm sure.", "start": 3167.367, "speaker": "SPEAKER_00"}, {"end": 3174.15, "text": "Ben, that was expected.", "start": 3171.889, "speaker": "SPEAKER_00"}, {"end": 3179.132, "text": "Are you worried about artificial general intelligence, AGI?", "start": 3174.55, "speaker": "SPEAKER_00"}, {"end": 3180.373, "text": "You are?", "start": 3180.093, "speaker": "SPEAKER_00"}, {"end": 3182.654, "text": "It's not my question.", "start": 3181.554, "speaker": "SPEAKER_00"}, {"end": 3184.475, "text": "You're answering first.", "start": 3182.714, "speaker": "SPEAKER_00"}, {"end": 3187.206, "text": " Well, I'm from Europe, we have to.", "start": 3186.046, "speaker": "SPEAKER_00"}, {"end": 3188.027, "text": "Yes, you are.", "start": 3187.487, "speaker": "SPEAKER_00"}, {"end": 3190.588, "text": "Don't worry, you've already regulated it.", "start": 3188.047, "speaker": "SPEAKER_00"}, {"end": 3191.148, "text": "It's fine.", "start": 3190.668, "speaker": "SPEAKER_00"}, {"end": 3193.068, "text": "Yeah, we have to protect the humanity and all.", "start": 3191.168, "speaker": "SPEAKER_00"}, {"end": 3196.709, "text": "The law has been signed, so it's no longer an issue.", "start": 3193.789, "speaker": "SPEAKER_02"}, {"end": 3197.17, "text": "You're welcome.", "start": 3196.749, "speaker": "SPEAKER_00"}, {"end": 3199.01, "text": "So what are you thinking?", "start": 3197.97, "speaker": "SPEAKER_00"}, {"end": 3199.59, "text": "No.", "start": 3199.47, "speaker": "SPEAKER_00"}, {"end": 3201.031, "text": "What do you mean no?", "start": 3200.311, "speaker": "SPEAKER_00"}, {"end": 3202.831, "text": "I mean, like, are you worried?", "start": 3201.851, "speaker": "SPEAKER_00"}, {"end": 3204.372, "text": "Oh, I can't wait.", "start": 3203.091, "speaker": "SPEAKER_00"}, {"end": 3208.633, "text": "That's a different way of looking at it.", "start": 3206.432, "speaker": "SPEAKER_00"}, {"end": 3214.595, "text": "If there's machines that can do all the crap work that I have to do all day and I can just mountain bike all day, like bring it on.", "start": 3209.033, "speaker": "SPEAKER_02"}, {"end": 3218.861, "text": " All right, well, let's stay hopeful then, I would say.", "start": 3216.112, "speaker": "SPEAKER_00"}, {"end": 3221.068, "text": "All right, thank you so much, Luc, for being with us.", "start": 3219.303, "speaker": "SPEAKER_00"}], "transcriptBySpeakers": [{"end": 10.384, "text": "the last session of the day. And we have Luke with us. Look, lurking in the back. Hello, Luke. Hello. So we both are green.", "start": 0.009, "speaker": "SPEAKER_00"}, {"end": 12.696, "text": "Yes, you finally caught up.", "start": 11.536, "speaker": "SPEAKER_02"}, {"end": 86.742, "text": "We finally like this is like you match the slides. So did we? Yeah, the color partially. Yeah, there is a bit of that. So AI, huh? Still a thing. I was I think it's a thing. Somebody mentioned it earlier today. Yeah. Yeah. So we've been having conversation about, uh, well, many things yesterday. And then of course the AI came along and, oh my, look is all in, like all in 100% in AI is this. So what do you do? Anybody actually working on a product that now embeds or integrates AI in some way? A few people? Okay. Who has given up on AI? Like AI is like so 2022. Okay. A few people, a few people, maybe, maybe look will change your mind today as well. Apparently, AI has eaten, well, looks website. Don't give it away. I should. Okay. I will not. So, but let's see. I mean, again, looking at what we, not, not what we can do with the AI, but maybe what we should do with the AI and how it all works under the hood. And also just for you to know, of course, let me just drop, drop in back in here. So Lucas Karanti, Managing Director, Sato Whittle Hill Ventures, but also earlier was a product director at Google and spent a lot of time of his life with little things like web form design. Remember that? Yeah, I do. 2010. Irrelevant now.", "start": 12.716, "speaker": "SPEAKER_00"}, {"end": 91.682, "text": "And mobile? Yeah, mobile, also irrelevant.", "start": 89.241, "speaker": "SPEAKER_02"}, {"end": 100.024, "text": "Well, then, I don't know what to expect now. Well, AI, I guess. So, please, round of applause to Lukro Blewski.", "start": 93.082, "speaker": "SPEAKER_00"}, {"end": 2444.935, "text": "Okay, thank you for having me. And, uh, there we go. And all of my little AI guys. Now, the staff here is so on top of it because we blew up these balloons and the moment you guys walked out the door, they took them all away. And I had an elaborate scheme for all of us stomping on the balloon at the same time and making a big noise. And now that has completely fallen apart. Thank you. But we're going to try this. Can you stand up for a second? You've been sitting really long. Your butt's probably hurt. Your brains are fried. Okay, so instead of stomping on a balloon, just try to make a big stomp noise. Okay, we have to time it, all right? Ready? So I'm here to talk about how AI ate my website, right? And when we think about that title, you might have some scary thoughts, AI, eating a website. What could that be? That doesn't sound very friendly, right? You might imagine some scary things. Okay, so on the count of three, I'm gonna try to time this. I want you to stomp a really scary stomp, and I'm gonna bring in a very scary AI thing. Okay, you ready? One, two, three. Oh, you did it! You conjured it! So here's a giant computer brain eating up all these web pages. Ah! My little guys! Let it go! Anybody scared? Okay, you can sit down. Don't worry, all the little A guys, AI guys, we'll be back. Woo! Okay. So in truth, when you think about AI, by the way, it's not as emphatic as when everybody pops a balloon. But that was good. I like that. Thank you. In truth, though, when most people think about AI, they think about something like chat GPT. We saw a bunch of examples of people using prompt engineering through chat GPT. This is a large language model. You can ask it questions like, who am I? Who is Luke? So I don't really need an intro slide anymore, because all this stuff is just available. But it's not just my website that these giant AI models have sort of subsumed and eaten up, right? They've also eaten up the entire rest of the web. And so when you ask a question like, where did Luke go to school? It's going to respond to something that seems really likely, right? Because people who do product designs very likely that they actually went to CMU. Probably many of them did. But I did not. So I say, I think Luke went to UIC and it responds, you're right, I'm so sorry, but he went to grad school at CMU, which I did not. So then I can say, did I really go to CMU? You're like, of course not, you're right. So no problem with reinforcement learning in these models. They're very oriented to doing what you say, right? You're wrong. I apologize. You're wrong again. I apologize. So we can kind of get them into whatever state we want. And sometimes they'll veer off into weird places, which I'm not going to do because the intro to this talk was weird enough. But let's zoom out for a second and look at this relationship. So we have websites, like my little website, where probably that information about me existed or maybe existed on the SmashingConf website. And you've got lots and lots of these websites, right? And all these things get basically turned into training data for these immense models. And they then can generate a response based on all the things they've learned about. What's the most likely thing to come next? And then they get tuned in different directions with things like reinforcement learning. So hence why it assumes that I as a product designer went to CMU. But this whole concept of training giant machine brains on the totality of published content on the internet You know, that doesn't sound nice, right? It sounds kind of scary, hence my scary AI thing here, conjures these scary narratives. But it doesn't have to be this way, because if we stop thinking in terms of this monolithic giant AI brain, And we instead start thinking about all the things different AI models, large or small, can do. And then we start thinking about how they can actually help us create new user experiences, build products in different ways. We heard a little bit about those in the morning sessions. There's a lot of things these things can do with language. They can basically take speech, turn into text. They can take text, turn it into speech. They can transform languages from one to the other. With vision, they can identify objects, identify faces. They can segment stuff. There's this huge list of capabilities that we now have because of these very capable AI models. And what's different from a few years ago, we have always had, not always, but machine learning's been around for a long time. But the dramatic shift over the past maybe two-ish years is that these very, very capable models are now accessible to anyone, anywhere and everywhere. You can download them, you can run them from the cloud. And so all these capabilities are just flying all over the place and showing up in so many different places. So people are learning about all these capabilities and integrating them into lots of different places. And this is the way I really like to think about this change, as opposed to the scary AI brain. To me, it's much more like, what are the new ingredients that we currently have that we can use to rethink experiences, to do things we couldn't do before, right? To go from all of these different technologies to new capabilities, especially on the web. Because God bless Brad and his enthusiasm for the original principles of the web. The web is not a very cool place right now. You go there and you're just inudated with like 15 dialogues and subscriptions for newsletters and you can't even read the content and honestly the first thing you want to do is kind of leave. It's not very cool. But I think we can make it really cool. So with that in mind, I'm just gonna walk you through a whole bunch of details on putting these things to work. And the place where I've been putting them to work is the place that I can control the most, which is my little part of the worldwide web here. This is my website. I started this 28 years ago, so how many of you are less than 28 years old? Okay, yeah, that's what I thought. What can you do in 28 years of having a website so you have things to aspire to? Well, I've written 2,000 text articles, put up about 100-something videos, bunch of PDF presentations, made almost 2,000 images. Yes, you too can achieve such things. That's a lot of stuff. And the way you can make use of that stuff using a web browser or the internet today is mostly like two modalities. You can search for things, right? You come back with some results. You can browse around. But even just my little website has 500,000 written words, 6,000 minutes of video. So like, God bless you if you ever make your way even through part of that. Highly unlikely. But what if we didn't have to just do search and browse? What if we could actually mirror the real world? And the real world that we're in, Vitaly keeps reminding us about the Pac-Man, right? Talk to each other. What if you could just talk? And so you could ask me a question like, hey, Luke, how do I decide what metrics I should use for my app? And in the real world, what I would do is be like, hmm, what do I know about that? I'll think about maybe a bunch of the things I've written, maybe some of the talks I've given in the past. And then I, in my head, assemble an answer and really quickly respond to you. The human brain is an amazing thing, much more amazing than any of these AI models we have. But I would think about what I know and within, you know, fractions of a second be able to assemble it into a coherent response and just keep talking and answer you. That's pretty different than our options online. If you do search, how do I pick metrics from my app? Go fish through these links. If you're browsing around and you make your way to all the articles tagged with metrics, well, there's 243 of them. So again, good luck. But what if instead we can do what I'm saying? How do I pick metrics from my app? So go through everything I've written or said on the topic, put them together into a coherent response, cite the sources, so you can go even deeper or verify what I said. So in this case, this response, how do I pick metrics from my app? You could see if you put this in the little system, I'm going to show you. You get this answer. And one of it is coming from a talk called Mind the Gap from 1 Minute 17. one hour 17 minutes and 50 seconds you click on that and it will take you to that part of the video right there queued up and you can see on the slide in the background how do I decide what to measure so those citations are tied directly to where the information is coming from so This is what happens when a bunch of AI models effectively eat all the pieces of content that make up a website and spit up a very different interaction model. And the first question people always have is, well, how do you do that? Well, I'll get to that. But first, I want to look at what the experience is, and then I'll talk through the pieces of how it gets put together. Now, when you see this, I ask a question, you respond with an answer. Many people say, well, isn't this just a chatbot? What LLM did you use for this chatbot? Is this chat GPT? This is what most people start thinking. Why does chat keep showing up? Number one, chat is a paradigm that's extremely familiar to all of us. You all have probably spent way too much time in your messaging app, even today, some of you may be on it right now. So this whole paradigm of messaging people, very familiar, very ubiquitous in our lives. So it's quite easy to kind of get in and running with that. But the pattern of this is a series of alternating messages, right? I say something, someone replies, the list goes on, If a message runs long, the space for it, you know, you get what these things look like, right? But is that really a great interaction design pattern for communicating to a global super brain? Maybe. I don't know. Probably not, though. If you actually look and see some of the issues that come up with this type of pattern, and this is some research from Nielsen Norman, You know, it's not really optimal for an iterative conversation with a set of documents. So they did a bunch of tests on these large language model-based chat experiences. And in particular, people having to kind of scroll through these conversation threads to find things, remember what was what, kind of veer things in the right direction. And a quote on the bottom here I think is pretty meaningful, which is, this behavior is a significant point of friction for everybody. that they observed in these studies. So we could probably do a bit better. Now, to account for some of the things that they found in those studies, we made use of a different interaction pattern when creating the experience you saw a second ago. So instead of this back and forth chat, everything expands, what you'll find in this design is something that looks a little bit more like this. And so what does it actually look like? Basically, all the previous question and answer pairs get collapsed after you've messaged them or completed them. Then the current one that you're on will open up, show you lots of space for the answer. And you can kind of go back and forth within them. So let me just show you what this looks like rather than talking about diagrams. So here you can see, how does free shipping help? It's part of this conversation. There's the response. If I want to go back and see how do you provide real time feedback, then I open up that one. And how does free shipping help thing close up? So I can kind of access one question and answer pair at a time. And then I can focus on that specific question that I have and its response when I'm asking it. And the rest kind of just rolls up and gets out of the way. So you can much more easily scan, find the thing you were looking for before, and get back to it. And this interaction actually works a little bit more like a FAQ style pattern. You guys have probably seen these. So it's like a big ask thing, right? And then there's all these frequently asked questions that you can open up and expand. Pretty much most websites have this stuff. And so this is a nice example of when you start from the problem space, right? When you don't start with, oh, we need a chatbot. Let's implement a chatbot. And you think about, well, what are we trying to do? Well, people are asking questions. They're trying to get responses. It's tied to a specific set of information. Maybe there's a different interaction paradigm that makes a little bit more sense here. And this Q&A interface that I just showed you looks a little bit more like a FAQ pattern than a chatbot pattern. And I bring this up because too often the process of design starts with something like a competitive audit. Or let's go see what everybody else is doing, put it up on a wall, right? And then as soon as you do that, if you've ever ridden a bike and you're riding a bike and you look at a tree, where are you gonna go? You're gonna ride right into the tree, right? It's called target fixation. So as soon as you see that chatbot pattern on chatGPT, guess where your design is gonna go? Right into that pattern. And whether intentionally or not, you end up copying what these different things are doing instead of letting the problem space actually guide the solution. So in this case, starting with understanding that problem of people are asking questions, they need answers, there's going to be a lot of them, they're going to go back and try and reference things, they need a way to scan through stuff, we ended up a little bit more of a FAQ thing than a chatbot thing. So now there's this expandable conversational interface. You ask a question, response comes down, some citations come up. But for the purpose of this presentation, because these are big slides on a screen, I've simplified a few things here. So I actually want to show you two more things on how this interface works. And one important one is this concept of suggested content. So you could see here. there's a response and then immediately after the response there's a series of follow-on questions that are related to the question you just asked. The first couple are directly tied to some of the questions that show up there and then the other ones are broader and kind of expand to other things that you can ask. And these suggested questions are actually really critical in anything that is just this big open-ended Box. Because one of the big problems people have with these things is, what do I ask? I don't know. Well, you can ask anything. That's like saying, don't ask anything. Nobody knows. And so having something like this addresses the issue of what should I ask. And lots of people have that problem because something like 65 or 70% of all the questions that get asked in this interface come from these suggested questions, right? So they're very helpful for getting people started and moving them in the right direction. So we knew from the start these were important, but it took a bit to actually iterate through the designs to get to what you saw there. So at first we experimented with something like very explicit to trigger suggested questions, because we're like, oh, we don't want to show these all the time. Somebody should be able to kind of click on something and get those. So we started with this little light bulb, right? Give me an idea, get it? So you click the light bulb, bunch of suggested questions come up. But then you get the classic, nobody knows what that icon is, are they gonna click it? So then you try something like, what else can I ask? And that shows up. But in both cases, the burden is on people to ask for these things. So instead, here's a series of designs that actually just started dropping it right after every answer, you'd get these suggestions. And there was a couple iterations on this until finally, We ended up on the most obvious way you could possibly do this, which is the way design always works. It's obvious in retrospect. Just put it directly in line right after the answer. Make it look almost like the answers, except a tiny bit different and away you go. So these follow directly in line in the interface. And the other important detail that I'm not showing, which is something we just launched a week ago, maybe two weeks ago, is now we created this new capability and you have to like squint the CDC is little like ones and twos popping up over there. These now are direct citations of the part of the answer that relates to where that content came from. So you can kind of see within the response which information piece is exciting. And if you click on them, as I'll show you in a second, you can go and see that information. So further kind of making obvious the way this thing works. And the way this works is it's pulling all this information from my website, right? All those articles, all those videos, all those audio files, and using that information to create a bunch of responses. for people's questions about design. Now, if you see one of those little citations, you can click on them and it'll expand it directly in line. So to go back to my diagram before here, how does free shipping help? You can see it's saying this article on evolving e-commerce shipping was used to make this answer. You can open up and there's the article and you have the ability to ask questions just of that article or go back to this more global mode. So what kinds of things can answer questions is not just those articles. As I mentioned before, there's videos, podcasts, PDFs, images, and more. But how do you get this stuff to be usable in this kind of interface? You have web pages, they have PDFs, you have websites, they've got images, you've got all this stuff. How do we take those things and then use them in this type of interface? So there's a couple things that we need to do and this is where the ability for you to use a whole bunch of different AI models starts to, I think, emerge. So, for example, if we look at videos, we have like a video, like this talk, let's say, is recorded. There'll be a video file at the end of it. We can use a speech-to-text model to take that video file, turn it into text. We can use a diarization model to break up who's talking when, when there's the Q&A session. We can have different voices and split that up. We can generate summaries, keywords, and then we can use vision models to extract items in those video screenshots, people, that kind of thing. So we take these files, we run them through a bunch of AI tasks using a number of models, and then we have to do a bunch of cleanup of that content. And when I say cleanup, I mean like we make sure that it understands that Luke is Luke, right? So it kind of cleans up the entities. We add some metadata, like date, location, et cetera. And so now we have content that we can use to start answering people's questions that before was just a bunch of video files. And to show you how far you can go with this, I'll give you one example. One of the things that we wanted to do was actually get some nice looking thumbnails for these videos. As you can see here, they're kind of all over the place. And so what you can do pretty quickly is generate a bunch of thumbnails from a video. This is a pretty trivial script with something like FFMpeg or what have you. spit out a bunch of videos, and then I can manually just go through four or five of those videos, or up to ten or something like that, and pick a good looking thumbnail that I like. And then using that little set of data, here's a bunch of thumbnails, pick this one, here's a bunch of thumbnails, pick this one, as examples, we can fine tune a little AI model that for all the rest of the hundred something videos picks a thumbnail for us. So the next time it goes through here, it's going to use this lightweight training that we give to pick this beautiful image of me here with the Which Paris Girl Are You screenshot. It nailed that part. It left me with a dumb little grin, but whatever. It's probably the best thumbnail of the bunch. So this is just a quick example of how you can think about having AI models do a lot of things for you. If you can kind of model a behavior to use that word intentionally, and if you can show a machine how to do something, you can train that machine to start doing it. And that, I think, is a really big switch to go off in your head. All right, if I can show a machine how to do this, I can get the machine to do it pretty consistently, even little things like this. And so this opens up a lot of possibilities in terms of, again, how you can invert experiences because you can have machines do all this processing. For audio, we can do the same thing. Take a bunch of podcasts, take a bunch of audio interviews, turn them into text, diarize them, summarize them. Whatever you need, do a bit of processing to them. Again, same kind of thing for articles. Summarize them, extract keywords, generate suggested questions, all this good stuff. So we start with all of these files that make up your typical web page is, websites, audio, video, text. images, what have you. We use these different AI models to turn them into useful, common kinds of content. And then when we have everything in a common format, this is where some of the secret sauce happens. What we do is we take all those little bits of content, call them chunks, whatever you want to call them, and we basically put them into an embedding space. that puts related concepts to it. This isn't exactly how it works, but this is a good mental model to have. So, e-commerce-y things go over here. Mobile touchy things are over here. What have you? And in some cases, those are video pieces. In other cases, those are images. In other cases, those are text snippets. So all this stuff is in basically one big, you can think of it as a fuzzy search index or another way to think about it as a semantic search index where you're actually looking things up by their relative meaning as opposed to things like keywords or metadata look up. So I'll show a system diagram super quick just so folks get it like this always happens. But you start with your docs, you do a whole bunch of this cleaning, you stick them into this search index that organizes them again in a very fuzzy kind of semantic way. And then you need some way of searching against it. You can have multiple ways of searching against it. Back comes some results. You can rank those results. And then you take what you found and you basically use them to generate a response. You can use lots of different models if you want. You can use one model. You can come up with a number of answers, pick the best one, using another AI model if you get the idea. And then you get to a point where you've got the right information to answer people's questions. So we break all the things into little chunks. We stick them into this crazy vector space. And then when people ask questions, we go find the relevant pieces and assemble them into answers. It's AI models up and down. In fact, it goes even deeper. And hopefully, this is one thing that you take away from this. If you go down the rabbit hole, there's like an AI model everywhere in this kind of workflow. Because one of the important things to do is actually learn, right? You get somebody an answer, you want to make sure that your system gets smarter. So one of the things that we did a couple weeks ago is we took all of these lists of questions people ask, and then the list of corresponding documents that we used to answer them, and we trained a custom retriever. which then spits out a score for how good is this document in answering the specific question. And we did that with using that feedback that we were getting. And if you may say, well, why does that matter? If you just take some off-the-shelf AI model that sticks things into this embedding space, that kind of puts things in the vector space that you can later find it, those are just trained on general sets of sentences, like global internet stuff. So they'll be okay. at finding the pieces that are relevant in my content. But they're not going to be great. And if you make a custom one, these scores mean nothing to you except bigger is better. So that cross-encoder compared to just using an open AI model does a heck of a lot better. It's much, much better. So you can make your own little AI models that find relevant bits of content, that understand what kind of content you have, and what good content is for the specific kinds of questions that get asked on your sites. And the whole thing becomes a virtuous loop. So some of you may be thinking, oh my god, that's a lot of work. Why would we do this? Why wouldn't I just show GPT on my website? Well, if you ask ChadGPT a question like, why do so many designs look the same? You're going to get this generic world knowledge answer, right? It's going to be like, well, it's because of cognitive biases, legal and regulatory constraints, economic factors, the influence of trends and cultural influences. It just sounds like a lot of horse shit to me. So all that work I showed you actually allows us to deliver something that mirrors the way I would actually answer a question. Kind of like reflects my specific opinions of things. So instead, when you ask our interface, why do you so many designs look the same? I basically say, look, it's because designers are just looking what everybody else is doing and kind of copying it without thinking too much. very different kind of answer. And then the other thing you get is all these citations that I was highlighting earlier, where again, you've got a bunch of information you've published in whatever websites you have or whatever applications you have, you can reference that stuff, make sure that it's cited and allow people to go deeper into all of it. Now, when I say go deeper, it's worth showing a little bit of example of what that actually means. So we asked this question, why do so many designs look the same? And that answer comes from, I mentioned a few times, videos, audio, web pages, whatever. If you were to click on one of those videos, what we're going to actually do is inside this conversational interface, bring up that video. I showed you an example earlier. Take you to that part of the video where that answer is showing up inside of the video track, and then give you the ability to ask questions of it, give you a. Summary of that video give you a full transcript that you can search against, so you can click that transcript thing and look for whatever you want inside that video file. If it's an audio file, same thing, we'll just embed an audio player, give you again that transcript and summary, allow you to ask questions of these multi-hour long audio files right then and there. If it is an article, we'll put the article in there. If it's a PDF, same thing, you kind of get it. And so again, this is an example of how this interface, this conversational interface, has basically eaten all the stuff that previously made a website. Because now all these things appear in the conversation. You ask a question. Here's the video that answers it. Now you're watching that video. You ask that a question. Here's an audio file. Now you can be inside that audio file or inside that article or through a PDF. Now I said the word PDF and my body sort of cringed a little bit, because PDFs are special, as in very painfully special. When I was building this interface, we first started with blog articles, and then we added video, and then we added audio, and then we added PDF. And when we added something like 400-ish PDFs to here, you could see something changed pretty dramatically. So before the question you could ask, what is design? It would say something like, design is the art of gradually applying constraints. So I kind of like this answer over here. And then I added the 400 PDFs, and all of a sudden, This one PDF file from 2006 about dogma free design like just dominated the results. And now it says it is directed towards human beings with the aim of solving design is the fundamental soul of human creation. I'm like, what is this? This doesn't sound like me. Oh, what turned out is inside this PDF, there was like four screens of quotes from other designers about what is design. And so the model, the information retrieval model is like, hell yeah, look at this baby, right? And it's like. And so the answer just becomes like this regurgitation of a whole bunch of quotes about what is design. So that's not good, right? And the other thing that was happening was a lot of junk is kind of showing up as part of this content. So if you break down what's inside, a PDF is like markup language, right? It's got so much stuff in there about how things should look as opposed to what the content is. And so you just get pieces of content that are just crap. I mean, look at some of these things. 1,268, the content is zero nature. One, two, six, five, biography photo, quote, auto video in the press. So you're like, dear AI model, please answer the question with this. And surprising the thing works at all. It's actually a testament to how amazing these things that it works at all. So what we ended up doing is cleaning this up again, getting rid of a bunch of stuff that's crap and clogging up our answers, introducing a mechanism where we diversify the answer. So like, yo, yo, yo, you can't just use this one PDF even though it loves to have design quotes. Break it up a little bit. And when we did that, we actually got to this really nice balance answer for what is design. There's a PDF there, there's a video, there's two podcasts, there's a text file. And now it's starting to really represent the diversity of content and clean it up. So the answer got a heck of a lot better through cleaning up all this content and balancing what shows up when. And hopefully I'm getting through to you. You know, people ask, like, what chatbot did you use for this? What LLM is this? Is this Chad GPT? Who cares what it is? If you feed it crap, it's going to respond with crap, right? So you really need to take a lot of the document prep and the data pipelines and the cleanup and all this stuff super seriously. We're dealing in a world now where people are like, well, what am I going to do as a designer? You're going to do data cleanup. You laugh, but it's like, you know, content strategy, anybody remember that? That was like a hot button, I don't know, 20 years ago or something like that. It's a big part of it, and this becomes more like direction, right? Art direction, content direction, model direction, and steering. Somebody needs to have an opinion when I was showing you those two answers on what is design. Somebody needs to have an opinion on what is the answer that you want to deliver with the system. What should it be? What can't it be? What is not okay? What is okay? Now, we could do even better, because recent advances, and we did this about a month or two ago, in vision models have made it possible for us to totally change the game and what we can do from images and PDFs. So before, we spent a bunch of time trying to get models to do text extraction. We got pretty good at extracting all the real content from a file. But now what we can do is actually generate things that say something like, the chart illustrates the impact of A-B testing results from 2012 to 2019. The gray line represents this, suggesting a discrepancy between the positive outcomes indicated by A-B testing and actual business results achieved. What do you think is going to make your responses better? This one. Here's another example. Here's a screenshot from my Mind the Gap talk. This is what we had before. Good job getting all that content and image out of there. That's a pretty tough task. Did a good job. This is so much better. And so now what we do is we have a pipeline that creates this kind of content from every single screen in the PDF and from every single image. And when you then apply that to your embedding index and you change the content again, you once more have a very different set of answers for what is design. And in this case, I think it's getting better. Like every time I've done this, this answer has gotten better. Every time we've enriched the content and spent the time to make sure that what we're feeding in and how we're processing it is a lot better. So this is a process of constant adjustment. It's the same thing like people say, OK, I launched the website. I'm done. No, you're not. That's the beginning of it. Oh, I added a chat bot. I'm done. No, that's the beginning of it. So for PDFs, we need lots of cleaning, but now you can see that there's these, in addition to some of the text-based tasks, we've actually got some vision tasks to make sense of all that information and make it useful for these models. The other really cool thing that this enables is once you start processing images and understanding semantically what's in an image, you can actually start to respond with images and get multimodal. So now your website doesn't have to just like spit out text. It can actually do things like this. So I ask a question, what's the difference between AR and VR? And like literally, this image shows the spectrum from VR to AR. You can see that it creates a completely digital world while AR, it's not only putting the image in the right place in the response, but it's actually referencing it in its response. So this creates a whole new set of capabilities. Now you can start to respond with pages of PDFs, too, in the same way. It knows what's in that page of the PDF because we've spent all this time to process it and put it in the embedding index. We've trained a retriever to find it when it's most relevant, and then we can start to assemble answers that make a ton more sense. Okay, that's a lot of stuff. But hopefully, big picture, let's take a step back. You have website. Website have stuff in it. Yes? Okay, you all know website have stuff. Different kind of stuff. AI models on stuff. Very cool. Whole new ball game. And I think we're just scratching at the surface of this. By the way, this is why I have these little AI guys, because they're hustling through all these little AI models, trying to figure out what to do and how to use what thing, when, and how. Behind the scenes, they're using those language models, those vision models, everything. And so the end result here is not some... Hold on. And the end result here is not some... Scary AI thing, but instead it's a new way to use all this content, right? To kind of mirror real world experiences. You don't have to go through those 10 blue links and click on them and then find the part of the article that kind of relates to the thing that you're trying to go browse through long lists of things. You can just say, what you're asking, right? And so you could say something like, hey, what should designers do about AI? Yeah, it's going to reference some talks. It's going to give you an answer. And away you go. So these conversational interfaces, whatever we want to call them, or whatever they become in the future. I launched this service on my website. By the way, you can all, I should have said this again. You can all go play with it. Ask.LukeW.com. Go ahead, ask whatever questions you want. Don't crash it. But we've been doing this for almost a year and a half now. And like every couple months, there's some big interesting change where we can do new things. Now we're doing these inline citations. Now we're putting images in there. And the models are actually referencing the images. Next up, we're going to try doing that within inline video files. Who knows? Lots of stuff to do. So on behalf of me and all my little AI guys, Thank you. And I have one question for you. Are we going to actually find out what the bananas are for? Because they've been sitting there all day, and I'm just dying to know. OK. No, no, no. Thank you. That's fine.", "start": 100.044, "speaker": "SPEAKER_02"}, {"end": 2445.856, "text": "All right.", "start": 2445.516, "speaker": "SPEAKER_00"}, {"end": 2447.837, "text": "They're still here.", "start": 2447.157, "speaker": "SPEAKER_02"}, {"end": 2461.934, "text": "That's right. Toby, you want a banana? I mean, I'm not going to throw bananas at you. That's a bit disrespectful, I think. Thank you so much, Luke, for the insights. Can we switch? Oh, sure.", "start": 2448.037, "speaker": "SPEAKER_00"}, {"end": 2466.976, "text": "Am I in your chair? No, I just feel... I have to sit near the bananas after that comment.", "start": 2462.194, "speaker": "SPEAKER_02"}, {"end": 2491.647, "text": "Oh, this is my spot. Oh, they blend in, that's okay. So I'm wondering at this point, actually when you think about like a selection of a model, I mean you basically said that it doesn't matter as long as you have like the proper information architecture, content strategy in place and all. So you would say that there is basically no preference in terms of what you're going for?", "start": 2467.516, "speaker": "SPEAKER_00"}, {"end": 2521.031, "text": "It depends on what you're trying to do so if you want a general high-level Open-ended task a very large foundational model Claude GPT those things is going to be off the shelf what you want because it's post-trained if you want to do a very specific task That benefit of being very broadly trained to do anything is actually a negative because they're not specifically trained to do one particular thing a very particular way.", "start": 2492.808, "speaker": "SPEAKER_02"}, {"end": 2530.596, "text": "Right. So does it mean that then it's in some specific cases like an enterprise world where you want to kind of really optimize a specific task instead of going for large language models to go for small language models?", "start": 2521.231, "speaker": "SPEAKER_00"}, {"end": 2579.644, "text": "The architecture is usually a combination of multiple things. So you use smaller fine-tuned models for very specific tasks where you want to do something concrete. Let's talk about Brad's example earlier. If I want my particular pattern library, to be a very strong influence on the creation of components, then I'm going to create a bunch of training examples and I'm going to fine tune a model that's already good at writing code to write exactly my specific kind of code. But in that same pipeline, I may use something bigger to answer like very high level things like make a plan for what this web page should be. Right, so that is a very broad high-level world knowledge task and then like turn it into this code using this library is a very different kind of specific task I see use a smaller fine-tune model, right?", "start": 2531.176, "speaker": "SPEAKER_02"}, {"end": 2662.552, "text": "I'm also wondering just how come like actually those things like citations, right? And and some of those nice these features that you find that in perplexity and in Claude like just sharing the actual output being able to Reef kind of fork it like what you get into this is fantastic, but most AI experiences are not like that. So I'm actually wondering, I mean, maybe that's my personal experience, but I just want to share and I'm wondering what you think as well. I think over the last maybe three weeks, I have used Google once. Maybe just once. I mean, Google Maps is a different story, right? But I mean, like, like literally searching and going through links, really rare. But at the same time, over the last two months, I maybe used CHGPT once. So the only thing I'm paying for is perplexity, and I'm using it all the time. And I mean, like, all the time. And I get the reference. Perplexity employee. Perplexity? No, no. Perplexity stock owner. But I think for me, the reason why I'm saying that is that, to me, perplexity is doing this incredible job and kind of combining all these different patterns that you were also mentioning. But why is it not more like a norm? Why is the norm still just chatbot? Why are most web pages pieces of shit? Yeah, but, well, if you look at your GPT, right, it's, you know, over two years, it feels like it's better, but not better, if it makes sense.", "start": 2579.884, "speaker": "SPEAKER_00"}, {"end": 2694.176, "text": "My joke was not a joke, really, right? Like, if you look at the early stages of mobile apps, the vast majority of mobile apps at the beginning were moving web pages into a tiny little screen. They were pretty bad. They're trying to apply very prior patterns to a new medium, and it just takes a while to learn how to use that medium, what the experiences should be, and there's going to be more evolution beyond what I showed and what perplexity is doing, whatever. We're still super early in this stuff.", "start": 2664.253, "speaker": "SPEAKER_02"}, {"end": 2732.168, "text": "But at the same time, I feel like there is a bit, I mean, we had a very intense discussion about that yesterday. Because again, when you look at how AI is perceived outside of the US, it's very different. Like in Europe, whenever you start having conversations, there was a conference around you at AI. And then there is this sense of frustration, I guess, which would be the right word. And the reason for that is very simple, because many people are speaking about, you know, 2022 AI comes up, everybody's like, wow, that's magical. Now, everybody's like, when people speak about AI, it's like it feels fake or hallucinating or slow because you always have to wait for your GPT out.", "start": 2694.716, "speaker": "SPEAKER_00"}, {"end": 2734.249, "text": "You can't render hands. What's wrong?", "start": 2732.188, "speaker": "SPEAKER_02"}, {"end": 2751.759, "text": "Yeah, like all those things. But I mean, I'm just wondering, where are we now with the idea? Do we feel like it's like really groundbreaking innovation? Because everybody was expecting, it felt like a big breakthrough in health care and whatnot. Is it where we are? Or is it not? What is the state of it now?", "start": 2734.309, "speaker": "SPEAKER_00"}, {"end": 2755.1, "text": "I mean, I literally showed you a digital version of my brain.", "start": 2751.979, "speaker": "SPEAKER_02"}, {"end": 2759.402, "text": "Which is fantastic to see, but you also have it all documented neatly, I think.", "start": 2755.601, "speaker": "SPEAKER_00"}, {"end": 2764.145, "text": "We should have digital versions of your brain and everybody else's brains.", "start": 2760.223, "speaker": "SPEAKER_02"}, {"end": 2770.003, "text": "And then you have this... Then I'll know what you're doing with these.", "start": 2767.682, "speaker": "SPEAKER_00"}, {"end": 2776.307, "text": "Oh, yeah, yeah, yeah. That's right. That's right. But do you see... No, it's the same answer. We're still so early.", "start": 2770.784, "speaker": "SPEAKER_02"}, {"end": 2776.867, "text": "Yeah.", "start": 2776.687, "speaker": "SPEAKER_01"}, {"end": 2790.895, "text": "We're going to figure this out. Like, when the iPhone came out in 2007, was there a TikTok? Nope. It's like everybody's brain now fused to TikTok in a terrible way. Yeah.", "start": 2776.947, "speaker": "SPEAKER_02"}, {"end": 2793.019, "text": "Right.", "start": 2792.739, "speaker": "SPEAKER_00"}, {"end": 2857.23, "text": "So like those, it's just, you know, there's a ton of time which will put, so if you ask me, I think as I tried to illustrate there, you can basically put AI throughout your whole stack. From the user experience all the way down to how you store information right to like what your data structures are or no data structures whatsoever your data structures are just text and then you can basically build up and down from there like that's. Pretty crazy. Even in all these other previous kind of like major technology changes we had, we didn't have anything that is the entire stack all at once. Parts of the stack changed and that had massive repercussions on the industry. Web, mobile, you name it, cloud, more on the lower side of the stack. But now the whole stack is basically being rethought and changed. So I think the changes that we're gonna see of this, again, personally, I believe are gonna be more dramatic than those.", "start": 2793.079, "speaker": "SPEAKER_02"}, {"end": 2885.932, "text": "Actually, I should probably also cover some questions from the audience, by the way. We can talk about your questions. No, no, that's okay. We can have a conversation anyway. There was a question that came up quickly. Can you talk about content strategy for AI? What does it mean? How do you maintain your application model for not just existing, but, ooh, that just jumped. But new content, yeah. New content and something on that.", "start": 2859.584, "speaker": "SPEAKER_00"}, {"end": 2970.737, "text": "Fascinating topic. I don't think anybody really knows yet. The one thing I didn't show with the Ask Luke system that I can, if people are interested, is when questions come in that I don't have answers to inside my content corpus, but I actually have an answer in my head. I've gotten into this workflow where I built an admin system where I just press a button and talk. It records all that, like I don't have to write a blog article or anything like that, it just slurps it all up and that becomes part of the content corpus and it grows. And so behind the scenes over the past year and a half, I've generated something like 500 equivalent articles, which imagine publishing 500 blog articles in a year. You'd be crazy, right? It doesn't happen. But now, Question comes in, I know what people are asking about, and if I have an answer, I just hit record, talk, don't care about the format, don't care about the process. And so that is a very different way of thinking about content strategy, because it's such a precious thing before, and it's so structured. And now you can, again, basically use natural language as your data structure. I've personally found it a tough bit to flip in my head. For a long time, I was still like, well, I have to then format it into an article, and I have to remove the parts that are... No, you don't.", "start": 2886.292, "speaker": "SPEAKER_02"}, {"end": 2984.226, "text": "Drop it all in there and let the system figure it out. And when you think about this type of data consumption, that's another question that came up and processing. How does it impact document format structures? Are we moving away from PDFs for good?", "start": 2972.355, "speaker": "SPEAKER_00"}, {"end": 2992.334, "text": "No, because people still want PDFs as output, which is funny. One of the requests I have is like, can you turn that answer into a PDF?", "start": 2985.387, "speaker": "SPEAKER_02"}, {"end": 2997.527, "text": "Well, now we can turn it into audio podcast.", "start": 2995.566, "speaker": "SPEAKER_00"}, {"end": 3017.058, "text": "Yeah, turn it into a podcast. It's funny. I always, not always, but I have in the past made fun of Microsoft Copilot because Microsoft Copilot sits alongside Microsoft Office and it's always like, you're making a Word document. Do you want to turn it into a PowerPoint? And then it's like, you're making a PowerPoint. Do you want to turn it into a Word document?", "start": 2997.567, "speaker": "SPEAKER_02"}, {"end": 3036.117, "text": "What am I doing? Right? Pat is wondering how do you test AI for hallucination? We are running a financial application and we can't suggest inaccurate recommendations or insights. Accuracy is very critical. We haven't started working on AI yet because we're afraid of giving wrong suggestions to our users.", "start": 3018.458, "speaker": "SPEAKER_00"}, {"end": 3117.713, "text": "Yep, so the reason why I did this little Ask Luke project year and a half ago or whatever was because of that exact question. It's very difficult to assess, is this right or wrong? When I showed the answer there, where did Luke go to school? I know the answer. Do any of you? No. So how can you tell if the system is right or wrong? And the reason we built that system on my corpus is because I know all that content, and I could assess and verify what is wrong or right. So when you have that capability, then you can train the system to be like, that's the correct answer. That's the wrong answer. That's the correct answer. That's the wrong answer. I literally have an interface that our AI researcher made for me, which is like, vote on the response, which one is correct. And I have sat there for hours being like left, right, left, right, left, right. But that was, you know, a long time ago. Now I think the systems are a lot better. At that, and it's obviously an issue that everybody's aware of in the space. I think it's becoming less and less of a concern. And especially if you do the type of thing that I showed there, right? It's not foolproof, but the odds of things going off the rails are like this as opposed to... like this and it'll get better and better and they'll be more fine tuning that you can do it.", "start": 3036.437, "speaker": "SPEAKER_02"}, {"end": 3126.197, "text": "And so how much time did it take from the very beginning when you just had all the documents to more or less decent results?", "start": 3118.053, "speaker": "SPEAKER_00"}, {"end": 3165.426, "text": "This is the thing about AI. You can very quickly get to a compelling demo that's like 80% or 90% of what you want. Right and we saw a bunch of that today I think in some of the earlier presentations like oh look I type this into Blank and out pops a thing That's kind of what I want the next 10% is like 90% of the work right and that's true in other Domains as well, but it's particularly true in AI to stitch together something. That's working pretty good Super fast, very easy. To make something that's really robust and works for a long time, really long slog, lots of work.", "start": 3126.257, "speaker": "SPEAKER_02"}, {"end": 3193.068, "text": "Maybe final question, then of course we'll have more conversations at the party, I'm sure. Ben, that was expected. Are you worried about artificial general intelligence, AGI? You are? It's not my question. You're answering first. Well, I'm from Europe, we have to. Yes, you are. Don't worry, you've already regulated it. It's fine. Yeah, we have to protect the humanity and all.", "start": 3167.367, "speaker": "SPEAKER_00"}, {"end": 3196.709, "text": "The law has been signed, so it's no longer an issue.", "start": 3193.789, "speaker": "SPEAKER_02"}, {"end": 3208.633, "text": "You're welcome. So what are you thinking? No. What do you mean no? I mean, like, are you worried? Oh, I can't wait. That's a different way of looking at it.", "start": 3196.749, "speaker": "SPEAKER_00"}, {"end": 3214.595, "text": "If there's machines that can do all the crap work that I have to do all day and I can just mountain bike all day, like bring it on.", "start": 3209.033, "speaker": "SPEAKER_02"}, {"end": 3221.068, "text": "All right, well, let's stay hopeful then, I would say. All right, thank you so much, Luc, for being with us.", "start": 3216.112, "speaker": "SPEAKER_00"}], "transcriptByParagraphs": [{"end": 0, "text": "the last session of the day. And we have Luke with us. Look, lurking in the back. Hello, Luke. Hello. So we both are green.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Yes, you finally caught up.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We finally like this is like you match the slides. So did we? Yeah, the color partially. Yeah, there is a bit of that. So AI, huh? Still a thing. I was I think it's a thing. Somebody mentioned it earlier today. Yeah. Yeah. So we've been having conversation about, uh, well, many things yesterday. And then of course the AI came along and, oh my, look is all in, like all in 100% in AI is this. So what do you do? Anybody actually working on a product that now embeds or integrates AI in some way? A few people? Okay. Who has given up on AI? Like AI is like so 2022. Okay.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "A few people, a few people, maybe, maybe look will change your mind today as well. Apparently, AI has eaten, well, looks website. Don't give it away. I should. Okay. I will not. So, but let's see. I mean, again, looking at what we, not, not what we can do with the AI, but maybe what we should do with the AI and how it all works under the hood. And also just for you to know, of course, let me just drop, drop in back in here.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "So Lucas Karanti, Managing Director, Sato Whittle Hill Ventures, but also earlier was a product director at Google and spent a lot of time of his life with little things like web form design. Remember that? Yeah, I do. 2010. Irrelevant now.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "And mobile? Yeah, mobile, also irrelevant.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Well, then, I don't know what to expect now. Well, AI, I guess. So, please, round of applause to Lukro Blewski.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Okay, thank you for having me. And, uh, there we go. And all of my little AI guys. Now, the staff here is so on top of it because we blew up these balloons and the moment you guys walked out the door, they took them all away. And I had an elaborate scheme for all of us stomping on the balloon at the same time and making a big noise. And now that has completely fallen apart. Thank you. But we're going to try this. Can you stand up for a second? You've been sitting really long. Your butt's probably hurt. Your brains are fried.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Okay, so instead of stomping on a balloon, just try to make a big stomp noise. Okay, we have to time it, all right? Ready? So I'm here to talk about how AI ate my website, right? And when we think about that title, you might have some scary thoughts, AI, eating a website. What could that be? That doesn't sound very friendly, right? You might imagine some scary things. Okay, so on the count of three, I'm gonna try to time this. I want you to stomp a really scary stomp, and I'm gonna bring in a very scary AI thing. Okay, you ready? One, two, three. Oh, you did it!", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You conjured it! So here's a giant computer brain eating up all these web pages. Ah! My little guys! Let it go! Anybody scared? Okay, you can sit down. Don't worry, all the little A guys, AI guys, we'll be back. Woo! Okay. So in truth, when you think about AI, by the way, it's not as emphatic as when everybody pops a balloon. But that was good. I like that. Thank you. In truth, though, when most people think about AI, they think about something like chat GPT. We saw a bunch of examples of people using prompt engineering through chat GPT. This is a large language model.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You can ask it questions like, who am I? Who is Luke? So I don't really need an intro slide anymore, because all this stuff is just available. But it's not just my website that these giant AI models have sort of subsumed and eaten up, right? They've also eaten up the entire rest of the web. And so when you ask a question like, where did Luke go to school? It's going to respond to something that seems really likely, right? Because people who do product designs very likely that they actually went to CMU. Probably many of them did. But I did not.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So I say, I think Luke went to UIC and it responds, you're right, I'm so sorry, but he went to grad school at CMU, which I did not. So then I can say, did I really go to CMU? You're like, of course not, you're right. So no problem with reinforcement learning in these models. They're very oriented to doing what you say, right? You're wrong. I apologize. You're wrong again. I apologize. So we can kind of get them into whatever state we want. And sometimes they'll veer off into weird places, which I'm not going to do because the intro to this talk was weird enough.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But let's zoom out for a second and look at this relationship. So we have websites, like my little website, where probably that information about me existed or maybe existed on the SmashingConf website. And you've got lots and lots of these websites, right? And all these things get basically turned into training data for these immense models. And they then can generate a response based on all the things they've learned about. What's the most likely thing to come next? And then they get tuned in different directions with things like reinforcement learning.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So hence why it assumes that I as a product designer went to CMU. But this whole concept of training giant machine brains on the totality of published content on the internet You know, that doesn't sound nice, right? It sounds kind of scary, hence my scary AI thing here, conjures these scary narratives. But it doesn't have to be this way, because if we stop thinking in terms of this monolithic giant AI brain, And we instead start thinking about all the things different AI models, large or small, can do.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And then we start thinking about how they can actually help us create new user experiences, build products in different ways. We heard a little bit about those in the morning sessions. There's a lot of things these things can do with language. They can basically take speech, turn into text. They can take text, turn it into speech. They can transform languages from one to the other. With vision, they can identify objects, identify faces. They can segment stuff. There's this huge list of capabilities that we now have because of these very capable AI models.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And what's different from a few years ago, we have always had, not always, but machine learning's been around for a long time. But the dramatic shift over the past maybe two-ish years is that these very, very capable models are now accessible to anyone, anywhere and everywhere. You can download them, you can run them from the cloud. And so all these capabilities are just flying all over the place and showing up in so many different places. So people are learning about all these capabilities and integrating them into lots of different places.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And this is the way I really like to think about this change, as opposed to the scary AI brain. To me, it's much more like, what are the new ingredients that we currently have that we can use to rethink experiences, to do things we couldn't do before, right? To go from all of these different technologies to new capabilities, especially on the web. Because God bless Brad and his enthusiasm for the original principles of the web. The web is not a very cool place right now.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You go there and you're just inudated with like 15 dialogues and subscriptions for newsletters and you can't even read the content and honestly the first thing you want to do is kind of leave. It's not very cool. But I think we can make it really cool. So with that in mind, I'm just gonna walk you through a whole bunch of details on putting these things to work. And the place where I've been putting them to work is the place that I can control the most, which is my little part of the worldwide web here. This is my website.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "I started this 28 years ago, so how many of you are less than 28 years old? Okay, yeah, that's what I thought. What can you do in 28 years of having a website so you have things to aspire to? Well, I've written 2,000 text articles, put up about 100-something videos, bunch of PDF presentations, made almost 2,000 images. Yes, you too can achieve such things. That's a lot of stuff. And the way you can make use of that stuff using a web browser or the internet today is mostly like two modalities. You can search for things, right? You come back with some results. You can browse around.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But even just my little website has 500,000 written words, 6,000 minutes of video. So like, God bless you if you ever make your way even through part of that. Highly unlikely. But what if we didn't have to just do search and browse? What if we could actually mirror the real world? And the real world that we're in, Vitaly keeps reminding us about the Pac-Man, right? Talk to each other. What if you could just talk? And so you could ask me a question like, hey, Luke, how do I decide what metrics I should use for my app?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And in the real world, what I would do is be like, hmm, what do I know about that? I'll think about maybe a bunch of the things I've written, maybe some of the talks I've given in the past. And then I, in my head, assemble an answer and really quickly respond to you. The human brain is an amazing thing, much more amazing than any of these AI models we have. But I would think about what I know and within, you know, fractions of a second be able to assemble it into a coherent response and just keep talking and answer you. That's pretty different than our options online.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "If you do search, how do I pick metrics from my app? Go fish through these links. If you're browsing around and you make your way to all the articles tagged with metrics, well, there's 243 of them. So again, good luck. But what if instead we can do what I'm saying? How do I pick metrics from my app? So go through everything I've written or said on the topic, put them together into a coherent response, cite the sources, so you can go even deeper or verify what I said. So in this case, this response, how do I pick metrics from my app?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You could see if you put this in the little system, I'm going to show you. You get this answer.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And one of it is coming from a talk called Mind the Gap from 1 Minute 17. one hour 17 minutes and 50 seconds you click on that and it will take you to that part of the video right there queued up and you can see on the slide in the background how do I decide what to measure so those citations are tied directly to where the information is coming from so This is what happens when a bunch of AI models effectively eat all the pieces of content that make up a website and spit up a very different interaction model. And the first question people always have is, well, how do you do that?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Well, I'll get to that. But first, I want to look at what the experience is, and then I'll talk through the pieces of how it gets put together. Now, when you see this, I ask a question, you respond with an answer. Many people say, well, isn't this just a chatbot? What LLM did you use for this chatbot? Is this chat GPT? This is what most people start thinking. Why does chat keep showing up? Number one, chat is a paradigm that's extremely familiar to all of us. You all have probably spent way too much time in your messaging app, even today, some of you may be on it right now.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So this whole paradigm of messaging people, very familiar, very ubiquitous in our lives. So it's quite easy to kind of get in and running with that. But the pattern of this is a series of alternating messages, right? I say something, someone replies, the list goes on, If a message runs long, the space for it, you know, you get what these things look like, right? But is that really a great interaction design pattern for communicating to a global super brain? Maybe. I don't know. Probably not, though.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "If you actually look and see some of the issues that come up with this type of pattern, and this is some research from Nielsen Norman, You know, it's not really optimal for an iterative conversation with a set of documents. So they did a bunch of tests on these large language model-based chat experiences. And in particular, people having to kind of scroll through these conversation threads to find things, remember what was what, kind of veer things in the right direction.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And a quote on the bottom here I think is pretty meaningful, which is, this behavior is a significant point of friction for everybody. that they observed in these studies. So we could probably do a bit better. Now, to account for some of the things that they found in those studies, we made use of a different interaction pattern when creating the experience you saw a second ago. So instead of this back and forth chat, everything expands, what you'll find in this design is something that looks a little bit more like this. And so what does it actually look like?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Basically, all the previous question and answer pairs get collapsed after you've messaged them or completed them. Then the current one that you're on will open up, show you lots of space for the answer. And you can kind of go back and forth within them. So let me just show you what this looks like rather than talking about diagrams. So here you can see, how does free shipping help? It's part of this conversation. There's the response. If I want to go back and see how do you provide real time feedback, then I open up that one. And how does free shipping help thing close up?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So I can kind of access one question and answer pair at a time. And then I can focus on that specific question that I have and its response when I'm asking it. And the rest kind of just rolls up and gets out of the way. So you can much more easily scan, find the thing you were looking for before, and get back to it. And this interaction actually works a little bit more like a FAQ style pattern. You guys have probably seen these. So it's like a big ask thing, right? And then there's all these frequently asked questions that you can open up and expand.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Pretty much most websites have this stuff. And so this is a nice example of when you start from the problem space, right? When you don't start with, oh, we need a chatbot. Let's implement a chatbot. And you think about, well, what are we trying to do? Well, people are asking questions. They're trying to get responses. It's tied to a specific set of information. Maybe there's a different interaction paradigm that makes a little bit more sense here. And this Q&A interface that I just showed you looks a little bit more like a FAQ pattern than a chatbot pattern.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And I bring this up because too often the process of design starts with something like a competitive audit. Or let's go see what everybody else is doing, put it up on a wall, right? And then as soon as you do that, if you've ever ridden a bike and you're riding a bike and you look at a tree, where are you gonna go? You're gonna ride right into the tree, right? It's called target fixation. So as soon as you see that chatbot pattern on chatGPT, guess where your design is gonna go? Right into that pattern.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And whether intentionally or not, you end up copying what these different things are doing instead of letting the problem space actually guide the solution. So in this case, starting with understanding that problem of people are asking questions, they need answers, there's going to be a lot of them, they're going to go back and try and reference things, they need a way to scan through stuff, we ended up a little bit more of a FAQ thing than a chatbot thing. So now there's this expandable conversational interface. You ask a question, response comes down, some citations come up.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But for the purpose of this presentation, because these are big slides on a screen, I've simplified a few things here. So I actually want to show you two more things on how this interface works. And one important one is this concept of suggested content. So you could see here. there's a response and then immediately after the response there's a series of follow-on questions that are related to the question you just asked. The first couple are directly tied to some of the questions that show up there and then the other ones are broader and kind of expand to other things that you can ask.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And these suggested questions are actually really critical in anything that is just this big open-ended Box. Because one of the big problems people have with these things is, what do I ask? I don't know. Well, you can ask anything. That's like saying, don't ask anything. Nobody knows. And so having something like this addresses the issue of what should I ask. And lots of people have that problem because something like 65 or 70% of all the questions that get asked in this interface come from these suggested questions, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So they're very helpful for getting people started and moving them in the right direction. So we knew from the start these were important, but it took a bit to actually iterate through the designs to get to what you saw there. So at first we experimented with something like very explicit to trigger suggested questions, because we're like, oh, we don't want to show these all the time. Somebody should be able to kind of click on something and get those. So we started with this little light bulb, right? Give me an idea, get it?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So you click the light bulb, bunch of suggested questions come up. But then you get the classic, nobody knows what that icon is, are they gonna click it? So then you try something like, what else can I ask? And that shows up. But in both cases, the burden is on people to ask for these things. So instead, here's a series of designs that actually just started dropping it right after every answer, you'd get these suggestions. And there was a couple iterations on this until finally, We ended up on the most obvious way you could possibly do this, which is the way design always works.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's obvious in retrospect. Just put it directly in line right after the answer. Make it look almost like the answers, except a tiny bit different and away you go. So these follow directly in line in the interface. And the other important detail that I'm not showing, which is something we just launched a week ago, maybe two weeks ago, is now we created this new capability and you have to like squint the CDC is little like ones and twos popping up over there. These now are direct citations of the part of the answer that relates to where that content came from.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So you can kind of see within the response which information piece is exciting. And if you click on them, as I'll show you in a second, you can go and see that information. So further kind of making obvious the way this thing works. And the way this works is it's pulling all this information from my website, right? All those articles, all those videos, all those audio files, and using that information to create a bunch of responses. for people's questions about design. Now, if you see one of those little citations, you can click on them and it'll expand it directly in line.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So to go back to my diagram before here, how does free shipping help? You can see it's saying this article on evolving e-commerce shipping was used to make this answer. You can open up and there's the article and you have the ability to ask questions just of that article or go back to this more global mode. So what kinds of things can answer questions is not just those articles. As I mentioned before, there's videos, podcasts, PDFs, images, and more. But how do you get this stuff to be usable in this kind of interface?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You have web pages, they have PDFs, you have websites, they've got images, you've got all this stuff. How do we take those things and then use them in this type of interface? So there's a couple things that we need to do and this is where the ability for you to use a whole bunch of different AI models starts to, I think, emerge. So, for example, if we look at videos, we have like a video, like this talk, let's say, is recorded. There'll be a video file at the end of it. We can use a speech-to-text model to take that video file, turn it into text.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We can use a diarization model to break up who's talking when, when there's the Q&A session. We can have different voices and split that up. We can generate summaries, keywords, and then we can use vision models to extract items in those video screenshots, people, that kind of thing. So we take these files, we run them through a bunch of AI tasks using a number of models, and then we have to do a bunch of cleanup of that content. And when I say cleanup, I mean like we make sure that it understands that Luke is Luke, right? So it kind of cleans up the entities.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We add some metadata, like date, location, et cetera. And so now we have content that we can use to start answering people's questions that before was just a bunch of video files. And to show you how far you can go with this, I'll give you one example. One of the things that we wanted to do was actually get some nice looking thumbnails for these videos. As you can see here, they're kind of all over the place. And so what you can do pretty quickly is generate a bunch of thumbnails from a video.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "This is a pretty trivial script with something like FFMpeg or what have you. spit out a bunch of videos, and then I can manually just go through four or five of those videos, or up to ten or something like that, and pick a good looking thumbnail that I like. And then using that little set of data, here's a bunch of thumbnails, pick this one, here's a bunch of thumbnails, pick this one, as examples, we can fine tune a little AI model that for all the rest of the hundred something videos picks a thumbnail for us.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So the next time it goes through here, it's going to use this lightweight training that we give to pick this beautiful image of me here with the Which Paris Girl Are You screenshot. It nailed that part. It left me with a dumb little grin, but whatever. It's probably the best thumbnail of the bunch. So this is just a quick example of how you can think about having AI models do a lot of things for you. If you can kind of model a behavior to use that word intentionally, and if you can show a machine how to do something, you can train that machine to start doing it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And that, I think, is a really big switch to go off in your head. All right, if I can show a machine how to do this, I can get the machine to do it pretty consistently, even little things like this. And so this opens up a lot of possibilities in terms of, again, how you can invert experiences because you can have machines do all this processing. For audio, we can do the same thing. Take a bunch of podcasts, take a bunch of audio interviews, turn them into text, diarize them, summarize them. Whatever you need, do a bit of processing to them. Again, same kind of thing for articles.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Summarize them, extract keywords, generate suggested questions, all this good stuff. So we start with all of these files that make up your typical web page is, websites, audio, video, text. images, what have you. We use these different AI models to turn them into useful, common kinds of content. And then when we have everything in a common format, this is where some of the secret sauce happens. What we do is we take all those little bits of content, call them chunks, whatever you want to call them, and we basically put them into an embedding space. that puts related concepts to it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "This isn't exactly how it works, but this is a good mental model to have. So, e-commerce-y things go over here. Mobile touchy things are over here. What have you? And in some cases, those are video pieces. In other cases, those are images. In other cases, those are text snippets. So all this stuff is in basically one big, you can think of it as a fuzzy search index or another way to think about it as a semantic search index where you're actually looking things up by their relative meaning as opposed to things like keywords or metadata look up.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So I'll show a system diagram super quick just so folks get it like this always happens. But you start with your docs, you do a whole bunch of this cleaning, you stick them into this search index that organizes them again in a very fuzzy kind of semantic way. And then you need some way of searching against it. You can have multiple ways of searching against it. Back comes some results. You can rank those results. And then you take what you found and you basically use them to generate a response. You can use lots of different models if you want. You can use one model.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You can come up with a number of answers, pick the best one, using another AI model if you get the idea. And then you get to a point where you've got the right information to answer people's questions. So we break all the things into little chunks. We stick them into this crazy vector space. And then when people ask questions, we go find the relevant pieces and assemble them into answers. It's AI models up and down. In fact, it goes even deeper. And hopefully, this is one thing that you take away from this.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "If you go down the rabbit hole, there's like an AI model everywhere in this kind of workflow. Because one of the important things to do is actually learn, right? You get somebody an answer, you want to make sure that your system gets smarter. So one of the things that we did a couple weeks ago is we took all of these lists of questions people ask, and then the list of corresponding documents that we used to answer them, and we trained a custom retriever. which then spits out a score for how good is this document in answering the specific question.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And we did that with using that feedback that we were getting. And if you may say, well, why does that matter? If you just take some off-the-shelf AI model that sticks things into this embedding space, that kind of puts things in the vector space that you can later find it, those are just trained on general sets of sentences, like global internet stuff. So they'll be okay. at finding the pieces that are relevant in my content. But they're not going to be great. And if you make a custom one, these scores mean nothing to you except bigger is better.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So that cross-encoder compared to just using an open AI model does a heck of a lot better. It's much, much better. So you can make your own little AI models that find relevant bits of content, that understand what kind of content you have, and what good content is for the specific kinds of questions that get asked on your sites. And the whole thing becomes a virtuous loop. So some of you may be thinking, oh my god, that's a lot of work. Why would we do this? Why wouldn't I just show GPT on my website? Well, if you ask ChadGPT a question like, why do so many designs look the same?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You're going to get this generic world knowledge answer, right? It's going to be like, well, it's because of cognitive biases, legal and regulatory constraints, economic factors, the influence of trends and cultural influences. It just sounds like a lot of horse shit to me. So all that work I showed you actually allows us to deliver something that mirrors the way I would actually answer a question. Kind of like reflects my specific opinions of things. So instead, when you ask our interface, why do you so many designs look the same?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "I basically say, look, it's because designers are just looking what everybody else is doing and kind of copying it without thinking too much. very different kind of answer. And then the other thing you get is all these citations that I was highlighting earlier, where again, you've got a bunch of information you've published in whatever websites you have or whatever applications you have, you can reference that stuff, make sure that it's cited and allow people to go deeper into all of it. Now, when I say go deeper, it's worth showing a little bit of example of what that actually means.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So we asked this question, why do so many designs look the same? And that answer comes from, I mentioned a few times, videos, audio, web pages, whatever. If you were to click on one of those videos, what we're going to actually do is inside this conversational interface, bring up that video. I showed you an example earlier. Take you to that part of the video where that answer is showing up inside of the video track, and then give you the ability to ask questions of it, give you a.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Summary of that video give you a full transcript that you can search against, so you can click that transcript thing and look for whatever you want inside that video file. If it's an audio file, same thing, we'll just embed an audio player, give you again that transcript and summary, allow you to ask questions of these multi-hour long audio files right then and there. If it is an article, we'll put the article in there. If it's a PDF, same thing, you kind of get it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so again, this is an example of how this interface, this conversational interface, has basically eaten all the stuff that previously made a website. Because now all these things appear in the conversation. You ask a question. Here's the video that answers it. Now you're watching that video. You ask that a question. Here's an audio file. Now you can be inside that audio file or inside that article or through a PDF. Now I said the word PDF and my body sort of cringed a little bit, because PDFs are special, as in very painfully special.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "When I was building this interface, we first started with blog articles, and then we added video, and then we added audio, and then we added PDF. And when we added something like 400-ish PDFs to here, you could see something changed pretty dramatically. So before the question you could ask, what is design? It would say something like, design is the art of gradually applying constraints. So I kind of like this answer over here. And then I added the 400 PDFs, and all of a sudden, This one PDF file from 2006 about dogma free design like just dominated the results.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And now it says it is directed towards human beings with the aim of solving design is the fundamental soul of human creation. I'm like, what is this? This doesn't sound like me. Oh, what turned out is inside this PDF, there was like four screens of quotes from other designers about what is design. And so the model, the information retrieval model is like, hell yeah, look at this baby, right? And it's like. And so the answer just becomes like this regurgitation of a whole bunch of quotes about what is design. So that's not good, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And the other thing that was happening was a lot of junk is kind of showing up as part of this content. So if you break down what's inside, a PDF is like markup language, right? It's got so much stuff in there about how things should look as opposed to what the content is. And so you just get pieces of content that are just crap. I mean, look at some of these things. 1,268, the content is zero nature. One, two, six, five, biography photo, quote, auto video in the press. So you're like, dear AI model, please answer the question with this. And surprising the thing works at all.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's actually a testament to how amazing these things that it works at all. So what we ended up doing is cleaning this up again, getting rid of a bunch of stuff that's crap and clogging up our answers, introducing a mechanism where we diversify the answer. So like, yo, yo, yo, you can't just use this one PDF even though it loves to have design quotes. Break it up a little bit. And when we did that, we actually got to this really nice balance answer for what is design. There's a PDF there, there's a video, there's two podcasts, there's a text file.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And now it's starting to really represent the diversity of content and clean it up. So the answer got a heck of a lot better through cleaning up all this content and balancing what shows up when. And hopefully I'm getting through to you. You know, people ask, like, what chatbot did you use for this? What LLM is this? Is this Chad GPT? Who cares what it is? If you feed it crap, it's going to respond with crap, right? So you really need to take a lot of the document prep and the data pipelines and the cleanup and all this stuff super seriously.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "We're dealing in a world now where people are like, well, what am I going to do as a designer? You're going to do data cleanup. You laugh, but it's like, you know, content strategy, anybody remember that? That was like a hot button, I don't know, 20 years ago or something like that. It's a big part of it, and this becomes more like direction, right? Art direction, content direction, model direction, and steering. Somebody needs to have an opinion when I was showing you those two answers on what is design.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Somebody needs to have an opinion on what is the answer that you want to deliver with the system. What should it be? What can't it be? What is not okay? What is okay? Now, we could do even better, because recent advances, and we did this about a month or two ago, in vision models have made it possible for us to totally change the game and what we can do from images and PDFs. So before, we spent a bunch of time trying to get models to do text extraction. We got pretty good at extracting all the real content from a file.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But now what we can do is actually generate things that say something like, the chart illustrates the impact of A-B testing results from 2012 to 2019. The gray line represents this, suggesting a discrepancy between the positive outcomes indicated by A-B testing and actual business results achieved. What do you think is going to make your responses better? This one. Here's another example. Here's a screenshot from my Mind the Gap talk. This is what we had before. Good job getting all that content and image out of there. That's a pretty tough task. Did a good job.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "This is so much better. And so now what we do is we have a pipeline that creates this kind of content from every single screen in the PDF and from every single image. And when you then apply that to your embedding index and you change the content again, you once more have a very different set of answers for what is design. And in this case, I think it's getting better. Like every time I've done this, this answer has gotten better. Every time we've enriched the content and spent the time to make sure that what we're feeding in and how we're processing it is a lot better.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "So this is a process of constant adjustment. It's the same thing like people say, OK, I launched the website. I'm done. No, you're not. That's the beginning of it. Oh, I added a chat bot. I'm done. No, that's the beginning of it. So for PDFs, we need lots of cleaning, but now you can see that there's these, in addition to some of the text-based tasks, we've actually got some vision tasks to make sense of all that information and make it useful for these models.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "The other really cool thing that this enables is once you start processing images and understanding semantically what's in an image, you can actually start to respond with images and get multimodal. So now your website doesn't have to just like spit out text. It can actually do things like this. So I ask a question, what's the difference between AR and VR? And like literally, this image shows the spectrum from VR to AR.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You can see that it creates a completely digital world while AR, it's not only putting the image in the right place in the response, but it's actually referencing it in its response. So this creates a whole new set of capabilities. Now you can start to respond with pages of PDFs, too, in the same way. It knows what's in that page of the PDF because we've spent all this time to process it and put it in the embedding index. We've trained a retriever to find it when it's most relevant, and then we can start to assemble answers that make a ton more sense. Okay, that's a lot of stuff.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But hopefully, big picture, let's take a step back. You have website. Website have stuff in it. Yes? Okay, you all know website have stuff. Different kind of stuff. AI models on stuff. Very cool. Whole new ball game. And I think we're just scratching at the surface of this. By the way, this is why I have these little AI guys, because they're hustling through all these little AI models, trying to figure out what to do and how to use what thing, when, and how. Behind the scenes, they're using those language models, those vision models, everything.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so the end result here is not some... Hold on. And the end result here is not some... Scary AI thing, but instead it's a new way to use all this content, right? To kind of mirror real world experiences. You don't have to go through those 10 blue links and click on them and then find the part of the article that kind of relates to the thing that you're trying to go browse through long lists of things. You can just say, what you're asking, right? And so you could say something like, hey, what should designers do about AI? Yeah, it's going to reference some talks.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's going to give you an answer. And away you go. So these conversational interfaces, whatever we want to call them, or whatever they become in the future. I launched this service on my website. By the way, you can all, I should have said this again. You can all go play with it. Ask.LukeW.com. Go ahead, ask whatever questions you want. Don't crash it. But we've been doing this for almost a year and a half now. And like every couple months, there's some big interesting change where we can do new things. Now we're doing these inline citations. Now we're putting images in there.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And the models are actually referencing the images. Next up, we're going to try doing that within inline video files. Who knows? Lots of stuff to do. So on behalf of me and all my little AI guys, Thank you. And I have one question for you. Are we going to actually find out what the bananas are for? Because they've been sitting there all day, and I'm just dying to know. OK. No, no, no. Thank you. That's fine.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "All right.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "They're still here.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "That's right. Toby, you want a banana? I mean, I'm not going to throw bananas at you. That's a bit disrespectful, I think. Thank you so much, Luke, for the insights. Can we switch? Oh, sure.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Am I in your chair? No, I just feel... I have to sit near the bananas after that comment.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Oh, this is my spot. Oh, they blend in, that's okay. So I'm wondering at this point, actually when you think about like a selection of a model, I mean you basically said that it doesn't matter as long as you have like the proper information architecture, content strategy in place and all. So you would say that there is basically no preference in terms of what you're going for?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "It depends on what you're trying to do so if you want a general high-level Open-ended task a very large foundational model Claude GPT those things is going to be off the shelf what you want because it's post-trained if you want to do a very specific task That benefit of being very broadly trained to do anything is actually a negative because they're not specifically trained to do one particular thing a very particular way.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Right. So does it mean that then it's in some specific cases like an enterprise world where you want to kind of really optimize a specific task instead of going for large language models to go for small language models?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "The architecture is usually a combination of multiple things. So you use smaller fine-tuned models for very specific tasks where you want to do something concrete. Let's talk about Brad's example earlier. If I want my particular pattern library, to be a very strong influence on the creation of components, then I'm going to create a bunch of training examples and I'm going to fine tune a model that's already good at writing code to write exactly my specific kind of code.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But in that same pipeline, I may use something bigger to answer like very high level things like make a plan for what this web page should be. Right, so that is a very broad high-level world knowledge task and then like turn it into this code using this library is a very different kind of specific task I see use a smaller fine-tune model, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "I'm also wondering just how come like actually those things like citations, right? And and some of those nice these features that you find that in perplexity and in Claude like just sharing the actual output being able to Reef kind of fork it like what you get into this is fantastic, but most AI experiences are not like that. So I'm actually wondering, I mean, maybe that's my personal experience, but I just want to share and I'm wondering what you think as well. I think over the last maybe three weeks, I have used Google once. Maybe just once.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "I mean, Google Maps is a different story, right? But I mean, like, like literally searching and going through links, really rare. But at the same time, over the last two months, I maybe used CHGPT once. So the only thing I'm paying for is perplexity, and I'm using it all the time. And I mean, like, all the time. And I get the reference. Perplexity employee. Perplexity? No, no. Perplexity stock owner.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "But I think for me, the reason why I'm saying that is that, to me, perplexity is doing this incredible job and kind of combining all these different patterns that you were also mentioning. But why is it not more like a norm? Why is the norm still just chatbot? Why are most web pages pieces of shit? Yeah, but, well, if you look at your GPT, right, it's, you know, over two years, it feels like it's better, but not better, if it makes sense.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "My joke was not a joke, really, right? Like, if you look at the early stages of mobile apps, the vast majority of mobile apps at the beginning were moving web pages into a tiny little screen. They were pretty bad. They're trying to apply very prior patterns to a new medium, and it just takes a while to learn how to use that medium, what the experiences should be, and there's going to be more evolution beyond what I showed and what perplexity is doing, whatever. We're still super early in this stuff.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "But at the same time, I feel like there is a bit, I mean, we had a very intense discussion about that yesterday. Because again, when you look at how AI is perceived outside of the US, it's very different. Like in Europe, whenever you start having conversations, there was a conference around you at AI. And then there is this sense of frustration, I guess, which would be the right word. And the reason for that is very simple, because many people are speaking about, you know, 2022 AI comes up, everybody's like, wow, that's magical.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Now, everybody's like, when people speak about AI, it's like it feels fake or hallucinating or slow because you always have to wait for your GPT out.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "You can't render hands. What's wrong?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Yeah, like all those things. But I mean, I'm just wondering, where are we now with the idea? Do we feel like it's like really groundbreaking innovation? Because everybody was expecting, it felt like a big breakthrough in health care and whatnot. Is it where we are? Or is it not? What is the state of it now?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "I mean, I literally showed you a digital version of my brain.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Which is fantastic to see, but you also have it all documented neatly, I think.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "We should have digital versions of your brain and everybody else's brains.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And then you have this... Then I'll know what you're doing with these.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Oh, yeah, yeah, yeah. That's right. That's right. But do you see... No, it's the same answer. We're still so early.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Yeah.", "start": 0, "speaker": "SPEAKER_01"}, {"end": 0, "text": "We're going to figure this out. Like, when the iPhone came out in 2007, was there a TikTok? Nope. It's like everybody's brain now fused to TikTok in a terrible way. Yeah.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Right.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "So like those, it's just, you know, there's a ton of time which will put, so if you ask me, I think as I tried to illustrate there, you can basically put AI throughout your whole stack. From the user experience all the way down to how you store information right to like what your data structures are or no data structures whatsoever your data structures are just text and then you can basically build up and down from there like that's. Pretty crazy. Even in all these other previous kind of like major technology changes we had, we didn't have anything that is the entire stack all at once.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Parts of the stack changed and that had massive repercussions on the industry. Web, mobile, you name it, cloud, more on the lower side of the stack. But now the whole stack is basically being rethought and changed. So I think the changes that we're gonna see of this, again, personally, I believe are gonna be more dramatic than those.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Actually, I should probably also cover some questions from the audience, by the way. We can talk about your questions. No, no, that's okay. We can have a conversation anyway. There was a question that came up quickly. Can you talk about content strategy for AI? What does it mean? How do you maintain your application model for not just existing, but, ooh, that just jumped. But new content, yeah. New content and something on that.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Fascinating topic. I don't think anybody really knows yet. The one thing I didn't show with the Ask Luke system that I can, if people are interested, is when questions come in that I don't have answers to inside my content corpus, but I actually have an answer in my head. I've gotten into this workflow where I built an admin system where I just press a button and talk. It records all that, like I don't have to write a blog article or anything like that, it just slurps it all up and that becomes part of the content corpus and it grows.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so behind the scenes over the past year and a half, I've generated something like 500 equivalent articles, which imagine publishing 500 blog articles in a year. You'd be crazy, right? It doesn't happen. But now, Question comes in, I know what people are asking about, and if I have an answer, I just hit record, talk, don't care about the format, don't care about the process. And so that is a very different way of thinking about content strategy, because it's such a precious thing before, and it's so structured.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And now you can, again, basically use natural language as your data structure. I've personally found it a tough bit to flip in my head. For a long time, I was still like, well, I have to then format it into an article, and I have to remove the parts that are... No, you don't.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Drop it all in there and let the system figure it out. And when you think about this type of data consumption, that's another question that came up and processing. How does it impact document format structures? Are we moving away from PDFs for good?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "No, because people still want PDFs as output, which is funny. One of the requests I have is like, can you turn that answer into a PDF?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Well, now we can turn it into audio podcast.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Yeah, turn it into a podcast. It's funny. I always, not always, but I have in the past made fun of Microsoft Copilot because Microsoft Copilot sits alongside Microsoft Office and it's always like, you're making a Word document. Do you want to turn it into a PowerPoint? And then it's like, you're making a PowerPoint. Do you want to turn it into a Word document?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "What am I doing? Right? Pat is wondering how do you test AI for hallucination? We are running a financial application and we can't suggest inaccurate recommendations or insights. Accuracy is very critical. We haven't started working on AI yet because we're afraid of giving wrong suggestions to our users.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "Yep, so the reason why I did this little Ask Luke project year and a half ago or whatever was because of that exact question. It's very difficult to assess, is this right or wrong? When I showed the answer there, where did Luke go to school? I know the answer. Do any of you? No. So how can you tell if the system is right or wrong? And the reason we built that system on my corpus is because I know all that content, and I could assess and verify what is wrong or right. So when you have that capability, then you can train the system to be like, that's the correct answer.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "That's the wrong answer. That's the correct answer. That's the wrong answer. I literally have an interface that our AI researcher made for me, which is like, vote on the response, which one is correct. And I have sat there for hours being like left, right, left, right, left, right. But that was, you know, a long time ago. Now I think the systems are a lot better. At that, and it's obviously an issue that everybody's aware of in the space. I think it's becoming less and less of a concern. And especially if you do the type of thing that I showed there, right?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "It's not foolproof, but the odds of things going off the rails are like this as opposed to... like this and it'll get better and better and they'll be more fine tuning that you can do it.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "And so how much time did it take from the very beginning when you just had all the documents to more or less decent results?", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "This is the thing about AI. You can very quickly get to a compelling demo that's like 80% or 90% of what you want. Right and we saw a bunch of that today I think in some of the earlier presentations like oh look I type this into Blank and out pops a thing That's kind of what I want the next 10% is like 90% of the work right and that's true in other Domains as well, but it's particularly true in AI to stitch together something. That's working pretty good Super fast, very easy. To make something that's really robust and works for a long time, really long slog, lots of work.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Maybe final question, then of course we'll have more conversations at the party, I'm sure. Ben, that was expected. Are you worried about artificial general intelligence, AGI? You are? It's not my question. You're answering first. Well, I'm from Europe, we have to. Yes, you are. Don't worry, you've already regulated it. It's fine. Yeah, we have to protect the humanity and all.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "The law has been signed, so it's no longer an issue.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "You're welcome. So what are you thinking? No. What do you mean no? I mean, like, are you worried? Oh, I can't wait. That's a different way of looking at it.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "If there's machines that can do all the crap work that I have to do all day and I can just mountain bike all day, like bring it on.", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "All right, well, let's stay hopeful then, I would say. All right, thank you so much, Luc, for being with us.", "start": 0, "speaker": "SPEAKER_00"}]}], "openaiAnswer": "", "fromSuggestion": false, "backfillResources": [{"id": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "url": "https://www.lukew.com/ff/entry.asp?2080", "type": 1, "title": "Smashing Conf: Is Atomic Design Dead?", "summary": "Brad Frost, in his presentation at Smashing Conf New York, discussed the evolution of design systems from HTML and CSS to the current use of AI models. He emphasized the need for a balance between compliance to design systems and product needs, and criticized the industry for isolating design systems. Frost suggested the creation of a global design system with reusable components, and highlighted the potential of AI models in facilitating design system work, such as creating custom components and translating between different design systems. However, he also raised questions about the future roles of designers and developers in the face of rapidly advancing AI.", "entities": [], "metadata": {"tags": ["The More You Own, The More You Maintain", "Patterns in Pattern Breaking", "Personal Interface Definitions", "design systems", "patterns", "components"], "title": "LukeW | Smashing Conf: Is Atomic Design Dead?", "author": "Luke Wroblewski", "longDate": "October 8, 2024", "shortDate": "Oct 8, 2024"}, "coverImage": "https://static.lukew.com/lukew_og_img8.png", "renderHtml": "

\t\n\n

\n \n

by October 8, 2024

\n\n

In his Is Atomic Design Dead? presentation at Smashing Conf New York, Brad Frost discussed the history of design systems and today's situation especially in light of very capable AI models than can generate code and designs. Here's my notes on his talk.

\n\n

Websites started as HTML and CSS. People began to design websites in Photoshop and as the number of Web sites and apps increased, the need for managing a brand and style across multiple platforms became clear. To manage this people turned to frameworks and component libraries which resulted in more frameworks and tools that eventually got integrated into design tools like Figma. It's been an ongoing expansion...
There's been lots of change over the years but at the highest level, we have design systems and products that use them to enforce brand, consistency, accessibility, and more.
Compliance to design systems pushes from one side and product needs push from the other. There needs to be a balance but currently the gap between the two is growing. A good balance is achieved through a virtuous cycle between product and systems.
The atomic design system tried to intentionally define use of atoms, molecules, organisms, templates, and pages to bridge the gap between the end state of a product and a design system.
As an industry, we went too far in resourcing design systems and making them a standalone thing within a company. They've been isolated.
Design system makers can't be insular. They need to reach out to product teams and work with them. They need to be helping product teams achieve their goals.
What if there were one global design system with common reusable components? Isn't that what HTML is for? Yes, but it's insufficient because we're still rebuilding date pickers everywhere.
Open UI tracks popular design systems and what's in them. It's a start to seeing what global component needs for the Web could look like.
Many pattern libraries ship with an aesthetic and people need to tweak it. A global design system should be very vanilla so you can style it as much as you want.
The Web still has an amazing scale of communication and collaboration. We need to rekindle the ideas of the early Web. We need to share and build together to get to a common freely usable design system.
AI models can help facilitate design system work. Today they do an OK job but in the future, fine-tuned models may create custom components on the fly. They can also translate between one design system and another or translate across programming languages.
This methodology could help companies translate existing and legacy code to new modern design systems. Likewise sketches or mockups could be quickly translated directly to design system components thereby speeding up processes.
Combining design system specifications with large language models allows you to steer AI generations more directly toward the right kind of code and components.
When product experiences are more dynamic (can be built on the fly), can we adapt them to individual preferences and needs? Like custom styles or interactions.
AI is now part of our design system toolkit and design systems are part of our AI toolkit.
But the rapid onset of AI also raises higher level questions about what designers and developers should be doing in the future? We're more than rectangle creators. We think and feel which differentiates us from just production level tasks. Use your brains, your intuition, and whole self to solve real problems.

\n\n\n\t\t\t\n\n

\n\n\n

", "textOnlyHtml": "by October 8, 2024\n\nIn his Is Atomic Design Dead? presentation at Smashing Conf New York, Brad Frost\ndiscussed the history of design systems and today's situation especially in\nlight of very capable AI models than can generate code and designs. Here's my\nnotes on his talk.\n\n * Websites started as HTML and CSS. People began to design websites in\n Photoshop and as the number of Web sites and apps increased, the need for\n managing a brand and style across multiple platforms became clear. To manage\n this people turned to frameworks and component libraries which resulted in\n more frameworks and tools that eventually got integrated into design tools\n like Figma. It's been an ongoing expansion...\n * There's been lots of change over the years but at the highest level, we have\n design systems and products that use them to enforce brand, consistency,\n accessibility, and more.\n * Compliance to design systems pushes from one side and product needs push from\n the other. There needs to be a balance but currently the gap between the two\n is growing. A good balance is achieved through a virtuous cycle between\n product and systems.\n * The atomic design system tried to intentionally define use of atoms,\n molecules, organisms, templates, and pages to bridge the gap between the end\n state of a product and a design system.\n * As an industry, we went too far in resourcing design systems and making them\n a standalone thing within a company. They've been isolated.\n * Design system makers can't be insular. They need to reach out to product\n teams and work with them. They need to be helping product teams achieve their\n goals.\n * What if there were one global design system with common reusable components?\n Isn't that what HTML is for? Yes, but it's insufficient because we're still\n rebuilding date pickers everywhere.\n * Open UI tracks popular design systems and what's in them. It's a start to\n seeing what global component needs for the Web could look like.\n * Many pattern libraries ship with an aesthetic and people need to tweak it. A\n global design system should be very vanilla so you can style it as much as\n you want.\n * The Web still has an amazing scale of communication and collaboration. We\n need to rekindle the ideas of the early Web. We need to share and build\n together to get to a common freely usable design system.\n * AI models can help facilitate design system work. Today they do an OK job but\n in the future, fine-tuned models may create custom components on the fly.\n They can also translate between one design system and another or translate\n across programming languages.\n * This methodology could help companies translate existing and legacy code to\n new modern design systems. Likewise sketches or mockups could be quickly\n translated directly to design system components thereby speeding up\n processes.\n * Combining design system specifications with large language models allows you\n to steer AI generations more directly toward the right kind of code and\n components.\n * When product experiences are more dynamic (can be built on the fly), can we\n adapt them to individual preferences and needs? Like custom styles or\n interactions.\n * AI is now part of our design system toolkit and design systems are part of\n our AI toolkit.\n * But the rapid onset of AI also raises higher level questions about what\n designers and developers should be doing in the future? We're more than\n rectangle creators. We think and feel which differentiates us from just\n production level tasks. Use your brains, your intuition, and whole self to\n solve real problems."}, {"id": "77e1b99b-7135-40ed-a347-e69985872667", "url": "https://static.lukew.com/video/Luke%20Wroblewski%20-%20Conversions@Google%202017.mp4", "date": 1493251200000, "type": 2, "title": "Mobile in The Future", "topics": ["mobile devices", "iPhone introduction", "Skype", "PayPal", "Cyber Monday", "Amazon Dash button", "natural user interfaces", "gesture-based user interfaces", "drop-down inputs", "search patterns", "hamburger menu", "Interactive devices", "machine learning", "Cyber Monday"], "endTime": 9918.98, "nameMap": {"SPEAKER_00": "Speaker 1", "SPEAKER_01": "Speaker 2", "SPEAKER_02": "Speaker 3", "SPEAKER_03": "Speaker 4", "SPEAKER_04": "Speaker 5", "SPEAKER_05": "Luke Wroblewski", "SPEAKER_06": "Speaker 6", "SPEAKER_07": "Speaker 7", "SPEAKER_08": "Speaker 8", "SPEAKER_09": "Speaker 9", "SPEAKER_10": "Speaker 10", "SPEAKER_11": "Speaker 11", "SPEAKER_12": "Speaker 12"}, "summary": "Luke Wroblewski discusses the rapid growth and impact of mobile technology since the iPhone's launch in 2007, emphasizing the importance of capitalizing on mobile opportunities, optimizing mobile payment and registration processes, and simplifying user experiences. They highlight the significance of natural user interfaces (NUI) in creating seamless, intuitive experiences and stress the need for fast, smooth mobile design. The speaker also addresses the challenges of understanding user intent and context in mobile interfaces and the importance of seamless device integration, dynamic browsing experiences, and A-B testing in design decisions. They envision a more integrated personal computing experience across multiple devices in the future.", "filename": "Luke%20Wroblewski%20-%20Conversions@Google%202017.mp4", "startTime": 9622.024, "coverImage": "https://qrmsyiqritqdkspkjoqy.supabase.co/storage/v1/object/public/lukew-media/batch-2-video-image/LukeWroblewskiConversionsGoogle2017_5.jpg", "transcript": [{"end": 3.913, "text": "Thank you.", "start": 2.532, "speaker": "SPEAKER_05"}, {"end": 7.136000000000001, "text": "I am here to stimulate you, it turns out.", "start": 3.913, "speaker": "SPEAKER_05"}, {"end": 8.338, "text": "I wish I had known that earlier.", "start": 7.136, "speaker": "SPEAKER_05"}, {"end": 10.6, "text": "I would have brought some props.", "start": 8.338, "speaker": "SPEAKER_05"}, {"end": 13.422, "text": "And actually, I do have one prop, which is audio.", "start": 10.6, "speaker": "SPEAKER_05"}, {"end": 17.066, "text": "So why don't I try to stimulate you with my audio really fast?", "start": 13.422, "speaker": "SPEAKER_05"}, {"end": 24.753, "text": "Tell me if this works or not.", "start": 17.066, "speaker": "SPEAKER_05"}, {"end": 25.634, "text": "A little bit.", "start": 24.753, "speaker": "SPEAKER_05"}, {"end": 26.575, "text": "Was it not loud enough?", "start": 25.634, "speaker": "SPEAKER_05"}, {"end": 30.073, "text": "Can we make it a little louder?", "start": 28.492, "speaker": "SPEAKER_05"}, {"end": 31.634, "text": "OK, let's do it again.", "start": 30.073, "speaker": "SPEAKER_05"}, {"end": 33.596, "text": "Tell me how stimulated this makes you.", "start": 31.634, "speaker": "SPEAKER_05"}, {"end": 33.836, "text": "Ready?", "start": 33.596, "speaker": "SPEAKER_05"}, {"end": 41.481, "text": "Yeah, a little bit.", "start": 33.836, "speaker": "SPEAKER_05"}, {"end": 43.403, "text": "All right.", "start": 41.481, "speaker": "SPEAKER_05"}, {"end": 47.266, "text": "So welcome to the future.", "start": 43.403, "speaker": "SPEAKER_05"}, {"end": 58.355, "text": "As I was preparing to talk to all of you today, I started pulling together a bunch of my thoughts around what's going on with mobile, where are we at, what kinds of things have we learned.", "start": 47.266, "speaker": "SPEAKER_05"}, {"end": 68.661, "text": "And I realized almost halfway through the process that I was going back to a bunch of examples that I had pulled together about six or so years ago.", "start": 58.355, "speaker": "SPEAKER_05"}, {"end": 72.263, "text": "And I was kind of revisiting these examples and saying, well, what has happened?", "start": 68.661, "speaker": "SPEAKER_05"}, {"end": 74.905, "text": "Have we made progress here?", "start": 72.263, "speaker": "SPEAKER_05"}, {"end": 78.147, "text": "And so that original talk was called Mobile to the Future.", "start": 74.905, "speaker": "SPEAKER_05"}, {"end": 86.253, "text": "And I sort of speculated, where will we get with some of these new technologies and capabilities that we have with mobile devices?", "start": 78.867, "speaker": "SPEAKER_05"}, {"end": 92.958, "text": "So when I realized I was revisiting a lot of these same topics, I thought, well, what better title change than mobile in the future?", "start": 86.253, "speaker": "SPEAKER_05"}, {"end": 96.681, "text": "And we'll see if it actually makes sense as we go here.", "start": 92.958, "speaker": "SPEAKER_05"}, {"end": 100.885, "text": "But what I want to do today, we actually have Two parts together.", "start": 96.681, "speaker": "SPEAKER_05"}, {"end": 103.592, "text": "There is part one that we are in right now.", "start": 100.885, "speaker": "SPEAKER_05"}, {"end": 108.224, "text": "And part one is effectively me talking to you about things that I find interesting.", "start": 103.592, "speaker": "SPEAKER_05"}, {"end": 111.353, "text": "We'll talk about the current state of mobile.", "start": 108.952, "speaker": "SPEAKER_05"}, {"end": 116.814, "text": "We'll talk about optimizing for that current state of mobile in areas that matter for conversion.", "start": 111.353, "speaker": "SPEAKER_05"}, {"end": 121.976, "text": "But then perhaps more importantly, I want to try and inspire us for what's next.", "start": 116.814, "speaker": "SPEAKER_05"}, {"end": 124.777, "text": "I want to talk about the future part of things.", "start": 121.976, "speaker": "SPEAKER_05"}, {"end": 128.278, "text": "Where can we go versus where are we?", "start": 124.777, "speaker": "SPEAKER_05"}, {"end": 134.78, "text": "So hopefully that'll give you a sense for how I think about some of these topics, the kinds of things we can discuss.", "start": 128.278, "speaker": "SPEAKER_05"}, {"end": 139.953, "text": "And most importantly, I hope it triggers a bunch of questions.", "start": 135.891, "speaker": "SPEAKER_05"}, {"end": 149.797, "text": "Because part two, we'll get a 30-minute break, but part two will be you can ask me any questions that you like, and hopefully I will be able to answer some of them.", "start": 139.953, "speaker": "SPEAKER_05"}, {"end": 155.139, "text": "Full disclosure, I have a slide deck with 618 slides in it.", "start": 149.797, "speaker": "SPEAKER_05"}, {"end": 162.042, "text": "So hopefully you ask me something in there, otherwise it's going to be a really awkward 90 minutes after the break.", "start": 155.139, "speaker": "SPEAKER_05"}, {"end": 164.163, "text": "OK, so part one.", "start": 162.042, "speaker": "SPEAKER_05"}, {"end": 167.246, "text": "10 years ago.", "start": 166.605, "speaker": "SPEAKER_05"}, {"end": 175.432, "text": "And I specifically use the 10 years ago moment because this year we hit a milestone.", "start": 167.246, "speaker": "SPEAKER_05"}, {"end": 188.722, "text": "On January 9, 2007, 10 years ago, I would say we had the moment that most people associate with the onset of the modern mobile era, the thing that kicked off the smartphone age.", "start": 175.432, "speaker": "SPEAKER_05"}, {"end": 191.584, "text": "And you're all probably pretty familiar with this event.", "start": 188.722, "speaker": "SPEAKER_05"}, {"end": 195.267, "text": "This was the introduction of the initial version of the iPhone.", "start": 191.584, "speaker": "SPEAKER_05"}, {"end": 198.192, "text": "And I think it makes a lot of sense.", "start": 196.912, "speaker": "SPEAKER_05"}, {"end": 203.314, "text": "I found myself referencing this moment a lot, because it's such a logical timestamp.", "start": 198.192, "speaker": "SPEAKER_05"}, {"end": 209.575, "text": "It's as good a moment as any to say, here's kind of where this mobile thing really kicked into high gear.", "start": 203.314, "speaker": "SPEAKER_05"}, {"end": 211.996, "text": "And by the way, I see poor people sitting on the stairs.", "start": 209.575, "speaker": "SPEAKER_05"}, {"end": 213.876, "text": "There's chairs up here.", "start": 211.996, "speaker": "SPEAKER_05"}, {"end": 219.277, "text": "You can enjoy comfy things for your bosom, as opposed to uncomfy things, if you so choose.", "start": 213.876, "speaker": "SPEAKER_05"}, {"end": 233.685, "text": "Kirk quoted this man a few times, I'll quote him one more time, and just kind of illustrating what was the intent behind the announcement of the iPhone.", "start": 222.957, "speaker": "SPEAKER_05"}, {"end": 242.772, "text": "And what Steve said is he wants to make a product that leapfrogs what's out there, that is way smarter than any mobile device has ever been.", "start": 233.685, "speaker": "SPEAKER_05"}, {"end": 256.287, "text": "And if we kind of start from this point, if we start from 10 years ago, I think we can actually look at some really interesting and hopefully eye-opening charts that illustrate just how far we've been.", "start": 244.666, "speaker": "SPEAKER_05"}, {"end": 274.389, "text": "And I'm going to say up front that I'm of the mindset that we underappreciate how much impact the past 10 years have had on our lives, especially in terms of distribution of technology and capabilities that technology brings.", "start": 257.259, "speaker": "SPEAKER_05"}, {"end": 276.83, "text": "So let me illustrate what I mean.", "start": 274.389, "speaker": "SPEAKER_05"}, {"end": 282.854, "text": "Steve says we want to make this leapfrog product that's way smarter than any device that's ever been out there.", "start": 276.83, "speaker": "SPEAKER_05"}, {"end": 287.487, "text": "So if we go back in time 10 years, which positions us towards the end of 2006.", "start": 282.854, "speaker": "SPEAKER_05"}, {"end": 292.389, "text": "At the end of 2006, there were actually smartphones in the world.", "start": 287.487, "speaker": "SPEAKER_05"}, {"end": 296.751, "text": "This wasn't a totally new invention from thin air.", "start": 292.389, "speaker": "SPEAKER_05"}, {"end": 303.755, "text": "In fact, the yearly total sales of smartphones about 10 years ago was around 68 million globally.", "start": 296.751, "speaker": "SPEAKER_05"}, {"end": 310.516, "text": "Now, that's only about 30% of the number of PCs, personal computers, that were sold that year.", "start": 305.053, "speaker": "SPEAKER_05"}, {"end": 316.96, "text": "It's about 239 million personal computers globally sold in 2006.", "start": 310.516, "speaker": "SPEAKER_05"}, {"end": 323.284, "text": "So smartphones, pretty small chunk of that, like I say, under 30%.", "start": 316.96, "speaker": "SPEAKER_05"}, {"end": 326.106, "text": "And what were these smartphones we talk about?", "start": 323.284, "speaker": "SPEAKER_05"}, {"end": 328.127, "text": "Does anybody remember what these looked like?", "start": 326.106, "speaker": "SPEAKER_05"}, {"end": 331.57, "text": "Let me remind us.", "start": 329.748, "speaker": "SPEAKER_05"}, {"end": 337.155, "text": "It's our old friend, the BlackBerry Pearl, the Motorola Q, the Palm Trail.", "start": 331.57, "speaker": "SPEAKER_05"}, {"end": 339.497, "text": "Anyone have one of these?", "start": 337.155, "speaker": "SPEAKER_05"}, {"end": 347.684, "text": "There was a little while there where I carried a BlackBerry Pearl around and an iPhone, because, you know, I don't know, that's what you used to do in Silicon Valley.", "start": 339.497, "speaker": "SPEAKER_05"}, {"end": 349.906, "text": "I don't know why.", "start": 347.684, "speaker": "SPEAKER_05"}, {"end": 354.591, "text": "This is the real thing, but I think this thing's awesome, so I'm going to carry it anyway.", "start": 349.906, "speaker": "SPEAKER_05"}, {"end": 356.27025925925926, "text": "We were all... ", "start": 355.523, "speaker": "SPEAKER_05"}, {"end": 358.045, "text": "Anyone remember the Crackberry Prayer?", "start": 356.27025925925926, "speaker": "SPEAKER_05"}, {"end": 359.546, "text": "Are you familiar with that?", "start": 358.045, "speaker": "SPEAKER_05"}, {"end": 368.6751794871795, "text": "So the Crackberry Prayer is when you're in a meeting and you're... ", "start": 359.546, "speaker": "SPEAKER_05"}, {"end": 370.174, "text": "True story.", "start": 368.6751794871795, "speaker": "SPEAKER_05"}, {"end": 371.915, "text": "Happened many times.", "start": 370.174, "speaker": "SPEAKER_05"}, {"end": 377.039, "text": "So this is kind of what the smartphones of 2006 looked like.", "start": 371.915, "speaker": "SPEAKER_05"}, {"end": 378.7, "text": "Pretty different from what we have today.", "start": 377.039, "speaker": "SPEAKER_05"}, {"end": 383.363, "text": "Let's look at what happened since.", "start": 381.882, "speaker": "SPEAKER_05"}, {"end": 385.865, "text": "This was the 10-year-ago situation.", "start": 383.363, "speaker": "SPEAKER_05"}, {"end": 388.947, "text": "First things first, I want to take a look at what happened with PCs.", "start": 385.865, "speaker": "SPEAKER_05"}, {"end": 399.281, "text": "I have to say before I show this chart, because I think you're going to be disappointed in this chart, the personal computer is the thing that helped get the internet into the world.", "start": 391.035, "speaker": "SPEAKER_05"}, {"end": 400.682, "text": "It's the thing that built Google.", "start": 399.281, "speaker": "SPEAKER_05"}, {"end": 413.091, "text": "It's the thing that built eBay and Facebook and so many of the services that are now ubiquitous in our lives and give us so much information and help and assistance throughout the day, let us do so much stuff.", "start": 400.682, "speaker": "SPEAKER_05"}, {"end": 420.137, "text": "So I really don't mean to diminish the importance of the PC when I show this little chart, but I'm going to do it anyway.", "start": 413.091, "speaker": "SPEAKER_05"}, {"end": 420.457, "text": "There you go.", "start": 420.137, "speaker": "SPEAKER_05"}, {"end": 429.485, "text": "Doesn't really look that impactful, does it?", "start": 426.303, "speaker": "SPEAKER_05"}, {"end": 432.587, "text": "Effectively, PCs kind of went up for a little while.", "start": 429.485, "speaker": "SPEAKER_05"}, {"end": 436.529, "text": "They peaked around 300-something million global shipments a year.", "start": 432.587, "speaker": "SPEAKER_05"}, {"end": 442.412, "text": "And then since then, they have been progressively falling after this plateau peak.", "start": 436.529, "speaker": "SPEAKER_05"}, {"end": 450.796, "text": "And if you look between 10 years ago and where we're at today, it's only a 30 million a year difference in terms of sales.", "start": 442.412, "speaker": "SPEAKER_05"}, {"end": 453.338, "text": "Not really the greatest growth rates out there, right?", "start": 450.796, "speaker": "SPEAKER_05"}, {"end": 467.595, "text": "So if that hasn't been such a huge success, and yet digital and the internet and everything is such a huge success, well, let's look at the smartphones number.", "start": 454.684, "speaker": "SPEAKER_05"}, {"end": 470.137, "text": "And here, by the way, tablets weren't so much a thing in 2006.", "start": 467.595, "speaker": "SPEAKER_05"}, {"end": 476.122, "text": "They're not that big a piece of this anyway, but I lumped them together to give you a sense of the overall scale.", "start": 470.137, "speaker": "SPEAKER_05"}, {"end": 479.384, "text": "We're now doing about 1.7 billion of these devices per year.", "start": 476.122, "speaker": "SPEAKER_05"}, {"end": 485.734, "text": "And some people, to me, have complained.", "start": 483.392, "speaker": "SPEAKER_05"}, {"end": 486.955, "text": "They're like, oh, it's flattening out.", "start": 485.734, "speaker": "SPEAKER_05"}, {"end": 491.199, "text": "I'm like, dude, there's only 7 billion people on the planet.", "start": 486.955, "speaker": "SPEAKER_05"}, {"end": 496.464, "text": "There's almost 2 billion of these things sold per year.", "start": 491.199, "speaker": "SPEAKER_05"}, {"end": 499.487, "text": "What do you want?", "start": 496.464, "speaker": "SPEAKER_05"}, {"end": 500.929, "text": "Yeah, it's totally ending.", "start": 499.487, "speaker": "SPEAKER_05"}, {"end": 502.91, "text": "Are you kidding?", "start": 500.929, "speaker": "SPEAKER_05"}, {"end": 503.351, "text": "2 billion.", "start": 502.91, "speaker": "SPEAKER_05"}, {"end": 511.177, "text": "So clearly, we've got a lot of growth in terms of devices when it comes to mobile.", "start": 506.234, "speaker": "SPEAKER_05"}, {"end": 516.02, "text": "And as I'm sure you recognize and understand, this creates a lot of opportunity.", "start": 511.177, "speaker": "SPEAKER_05"}, {"end": 524.645, "text": "Again, if you consider the PC, this thing is what created the digital economy that many of our industries were built off of, right?", "start": 516.02, "speaker": "SPEAKER_05"}, {"end": 528.067, "text": "This line had so much impact on our planet and our lives.", "start": 524.645, "speaker": "SPEAKER_05"}, {"end": 530.188, "text": "How much impact will a line like this have?", "start": 528.067, "speaker": "SPEAKER_05"}, {"end": 536.759, "text": "And this growth of devices, of course, leads to growth of use.", "start": 532.818, "speaker": "SPEAKER_05"}, {"end": 540.5, "text": "So let's go and rewind our clock one more time back to 2006.", "start": 536.759, "speaker": "SPEAKER_05"}, {"end": 546.462, "text": "And this time we're going to look at usage and audience sizes, not at device shipments.", "start": 540.5, "speaker": "SPEAKER_05"}, {"end": 551.863, "text": "Because maybe they're shipping all these devices and we just have filled the world with a lot of expensive doorstops, right?", "start": 546.462, "speaker": "SPEAKER_05"}, {"end": 566.442, "text": "Well, looking at Facebook's monthly active users all the way back in 2006 again, there was about 12 million desktop monthly active users.", "start": 553.413, "speaker": "SPEAKER_05"}, {"end": 575.109, "text": "10 years ago, or 10 years later, sorry, 10 years later, we're at 120 million desktop monthly active users.", "start": 566.442, "speaker": "SPEAKER_05"}, {"end": 580.818, "text": "And again, you can see a similar sort of rise and fall on the Facebook side.", "start": 575.109, "speaker": "SPEAKER_05"}, {"end": 584.499, "text": "And at this point, I'm sure you know what the other line is going to look like, but I'll show it to you anyway.", "start": 580.818, "speaker": "SPEAKER_05"}, {"end": 588.321, "text": "This is monthly active mobile users.", "start": 584.499, "speaker": "SPEAKER_05"}, {"end": 593.403, "text": "And that has reached about 1.7 billion people.", "start": 588.321, "speaker": "SPEAKER_05"}, {"end": 601.206, "text": "So it's a very similar pattern to what we saw with devices in general, which indicates to you these devices actually get used, unless they're desktops.", "start": 593.403, "speaker": "SPEAKER_05"}, {"end": 602.066, "text": "Sorry.", "start": 601.206, "speaker": "SPEAKER_05"}, {"end": 602.606, "text": "No, I'm kidding.", "start": 602.066, "speaker": "SPEAKER_05"}, {"end": 605.427, "text": "Desktops still get used and are important in different ways.", "start": 602.606, "speaker": "SPEAKER_05"}, {"end": 615.188, "text": "Pure quantity, right, mobile really dwarfs this experience, at least for Facebook's audience base.", "start": 608.365, "speaker": "SPEAKER_05"}, {"end": 620.23, "text": "And some people will say, okay, well, that's cool for Facebook, but I'm not Facebook.", "start": 615.188, "speaker": "SPEAKER_05"}, {"end": 624.632, "text": "Maybe these big established companies can really benefit from this device growth.", "start": 620.23, "speaker": "SPEAKER_05"}, {"end": 626.673, "text": "But what about new people entering the market?", "start": 624.632, "speaker": "SPEAKER_05"}, {"end": 635.637, "text": "And I think the impact of all of this mobile stuff isn't just felt by established big companies.", "start": 628.291, "speaker": "SPEAKER_05"}, {"end": 641.961, "text": "It actually creates amazing opportunity to reach a huge amount of people in a way that you previously couldn't.", "start": 635.637, "speaker": "SPEAKER_05"}, {"end": 644.523, "text": "So let's, again, rewind our clock.", "start": 641.961, "speaker": "SPEAKER_05"}, {"end": 650.187, "text": "And this time we're going to look at the fastest growing app ever up until 2005.", "start": 644.523, "speaker": "SPEAKER_05"}, {"end": 652.329, "text": "Anybody have an idea what that is?", "start": 650.187, "speaker": "SPEAKER_05"}, {"end": 658.715, "text": "What was the fastest growing app ever, roughly, 10, 12 years ago.", "start": 652.329, "speaker": "SPEAKER_05"}, {"end": 662.8060869565218, "text": "Okay, somebody over there said... ", "start": 658.715, "speaker": "SPEAKER_05"}, {"end": 666.0548913043478, "text": "Somebody over here said... ", "start": 662.8060869565218, "speaker": "SPEAKER_05"}, {"end": 675.32, "text": "Unfortunately, it's neither of those, but I remember those from the .com era.", "start": 666.0548913043478, "speaker": "SPEAKER_05"}, {"end": 679.2968837209303, "text": "I think I still hold stock in... ", "start": 675.32, "speaker": "SPEAKER_05"}, {"end": 685.684, "text": "The fastest growing app in 2005 was our friend Skype.", "start": 679.2968837209303, "speaker": "SPEAKER_05"}, {"end": 686.664, "text": "I knew that.", "start": 685.684, "speaker": "SPEAKER_05"}, {"end": 694.063, "text": "I have a six-year-old daughter who, every time we ever say anything, she's like, I was going to say that.", "start": 688.42, "speaker": "SPEAKER_05"}, {"end": 694.303, "text": "Really?", "start": 694.063, "speaker": "SPEAKER_05"}, {"end": 697.505, "text": "Well, why didn't you?", "start": 694.303, "speaker": "SPEAKER_05"}, {"end": 697.865, "text": "Always.", "start": 697.505, "speaker": "SPEAKER_05"}, {"end": 701.407, "text": "She was going to say it, no matter what it was.", "start": 697.865, "speaker": "SPEAKER_05"}, {"end": 705.849, "text": "So 630 days to hit 40 million users.", "start": 701.407, "speaker": "SPEAKER_05"}, {"end": 708.29, "text": "This was huge at the time, right?", "start": 705.849, "speaker": "SPEAKER_05"}, {"end": 711.492, "text": "Anybody remember how popular and excited people were about Skype?", "start": 708.29, "speaker": "SPEAKER_05"}, {"end": 716.614, "text": "Skype changed hands three times between eBay, private equity, Microsoft.", "start": 711.492, "speaker": "SPEAKER_05"}, {"end": 717.415, "text": "It was a really big deal.", "start": 716.614, "speaker": "SPEAKER_05"}, {"end": 724.832, "text": "And again, 630 days to reach 40 million users was a really big accomplishment.", "start": 718.75, "speaker": "SPEAKER_05"}, {"end": 729.174, "text": "Now, more recently, I'm going to show you another app.", "start": 724.832, "speaker": "SPEAKER_05"}, {"end": 735.096, "text": "This is a Super Mario Run on just one platform, only on iOS.", "start": 729.174, "speaker": "SPEAKER_05"}, {"end": 737.176, "text": "They just released it on Android, I think, yesterday.", "start": 735.096, "speaker": "SPEAKER_05"}, {"end": 739.277, "text": "I saw the headline on my way in.", "start": 737.176, "speaker": "SPEAKER_05"}, {"end": 740.578, "text": "It's now on Android.", "start": 739.277, "speaker": "SPEAKER_05"}, {"end": 744.799, "text": "So this release was only on one mobile operating system.", "start": 740.578, "speaker": "SPEAKER_05"}, {"end": 746.86, "text": "And it took four days.", "start": 744.799, "speaker": "SPEAKER_05"}, {"end": 748.918, "text": "to hit 40 million.", "start": 748.137, "speaker": "SPEAKER_05"}, {"end": 750.419, "text": "In fact, this line is BS.", "start": 748.918, "speaker": "SPEAKER_05"}, {"end": 752.68, "text": "I actually tried to give it a curve, but there's no curve.", "start": 750.419, "speaker": "SPEAKER_05"}, {"end": 757.744, "text": "There's just a straight line on this chart.", "start": 752.68, "speaker": "SPEAKER_05"}, {"end": 759.225, "text": "I'm lying to you, statistics.", "start": 757.744, "speaker": "SPEAKER_05"}, {"end": 762.407, "text": "But what a freaking difference, right?", "start": 759.225, "speaker": "SPEAKER_05"}, {"end": 763.628, "text": "630 days, and 10, 11 years later, four days.", "start": 762.407, "speaker": "SPEAKER_05"}, {"end": 773.194, "text": "And so this shows that even something new doesn't have to be this established.", "start": 768.39, "speaker": "SPEAKER_05"}, {"end": 782.482, "text": "Now granted, Nintendo, Mario, all these things, maybe it's an extreme example, but it just shows you how the velocity is increasing, the type of scale we now have to operate on.", "start": 773.194, "speaker": "SPEAKER_05"}, {"end": 787.044, "text": "Mobile is at the point where we've reached the state of deployment.", "start": 783.703, "speaker": "SPEAKER_05"}, {"end": 797.627, "text": "There's a platform, a user base, devices, capabilities, stuff that allows us to now take advantage of this amazing scale at which technology operates on the planet.", "start": 787.044, "speaker": "SPEAKER_05"}, {"end": 800.048, "text": "I'll talk about that a little bit more in a second.", "start": 797.627, "speaker": "SPEAKER_05"}, {"end": 807.61, "text": "But before I do that, I thought it would just be fun to talk about how long it took things to reach 10 million users.", "start": 800.048, "speaker": "SPEAKER_05"}, {"end": 809.311, "text": "So we talked about Super Mario Run.", "start": 807.61, "speaker": "SPEAKER_05"}, {"end": 813.092, "text": "It's about, I think, less than a day actually before it hit 10 million.", "start": 809.311, "speaker": "SPEAKER_05"}, {"end": 815.956, "text": "How about Pokemon Go?", "start": 814.795, "speaker": "SPEAKER_05"}, {"end": 816.657, "text": "Any estimates?", "start": 815.956, "speaker": "SPEAKER_05"}, {"end": 822.041, "text": "How long did it take Pokemon Go to hit 10 million users?", "start": 816.657, "speaker": "SPEAKER_05"}, {"end": 823.882, "text": "Any guesses?", "start": 822.041, "speaker": "SPEAKER_05"}, {"end": 824.603, "text": "Two hours?", "start": 823.882, "speaker": "SPEAKER_05"}, {"end": 828.046, "text": "OK.", "start": 824.603, "speaker": "SPEAKER_05"}, {"end": 829.647, "text": "More?", "start": 828.046, "speaker": "SPEAKER_05"}, {"end": 829.947, "text": "Again?", "start": 829.647, "speaker": "SPEAKER_05"}, {"end": 831.368, "text": "OK.", "start": 829.947, "speaker": "SPEAKER_05"}, {"end": 832.889, "text": "That's good.", "start": 831.368, "speaker": "SPEAKER_05"}, {"end": 835.892, "text": "It's not really a number, but I appreciate the effort.", "start": 832.889, "speaker": "SPEAKER_05"}, {"end": 840.255, "text": "Pokemon Go took two days.", "start": 835.892, "speaker": "SPEAKER_05"}, {"end": 840.956, "text": "OK.", "start": 840.255, "speaker": "SPEAKER_05"}, {"end": 843.317, "text": "Angry Birds Space.", "start": 840.956, "speaker": "SPEAKER_05"}, {"end": 843.818, "text": "Any guesses?", "start": 843.317, "speaker": "SPEAKER_05"}, {"end": 848.602, "text": "A week?", "start": 846.322, "speaker": "SPEAKER_05"}, {"end": 850.283, "text": "Originally, the numbers I got were 10 days.", "start": 848.602, "speaker": "SPEAKER_05"}, {"end": 856.704, "text": "Then the Angry Birds space guys contacted me and said, no, it was actually three days.", "start": 850.283, "speaker": "SPEAKER_05"}, {"end": 857.924, "text": "So I got to trust them.", "start": 856.704, "speaker": "SPEAKER_05"}, {"end": 860.305, "text": "I assume they know.", "start": 857.924, "speaker": "SPEAKER_05"}, {"end": 861.765, "text": "OK, here's a fun one.", "start": 860.305, "speaker": "SPEAKER_05"}, {"end": 868.207, "text": "Google+, how many days to hit 10 million users?", "start": 861.765, "speaker": "SPEAKER_05"}, {"end": 872.567, "text": "Nobody's going to get this one.", "start": 868.207, "speaker": "SPEAKER_05"}, {"end": 872.988, "text": "Seven?", "start": 872.567, "speaker": "SPEAKER_05"}, {"end": 873.5443636363636, "text": "Somebody guesses? ", "start": 872.988, "speaker": "SPEAKER_05"}, {"end": 873.668, "text": "200?", "start": 873.5443636363636, "speaker": "SPEAKER_05"}, {"end": 876.753, "text": "16.", "start": 876.653, "speaker": "SPEAKER_05"}, {"end": 876.993, "text": "OK.", "start": 876.753, "speaker": "SPEAKER_05"}, {"end": 877.254, "text": "Yay.", "start": 876.993, "speaker": "SPEAKER_05"}, {"end": 880.696, "text": "Twitter.", "start": 877.254, "speaker": "SPEAKER_05"}, {"end": 881.737, "text": "Any guesses on Twitter?", "start": 880.696, "speaker": "SPEAKER_05"}, {"end": 881.777, "text": "20.", "start": 881.737, "speaker": "SPEAKER_05"}, {"end": 882.838, "text": "Just a bit off.", "start": 881.777, "speaker": "SPEAKER_05"}, {"end": 885.36, "text": "780.", "start": 882.838, "speaker": "SPEAKER_05"}, {"end": 885.6, "text": "Facebook?", "start": 885.36, "speaker": "SPEAKER_05"}, {"end": 885.64, "text": "300?", "start": 885.6, "speaker": "SPEAKER_05"}, {"end": 890.563, "text": "At this point, you should note that my chart is going down.", "start": 885.64, "speaker": "SPEAKER_05"}, {"end": 909.379, "text": "Just saying.", "start": 906.577, "speaker": "SPEAKER_05"}, {"end": 912.182, "text": "852 days for Facebook.", "start": 909.379, "speaker": "SPEAKER_05"}, {"end": 916.666, "text": "And so, again, it just sort of shows you how things are accelerating, right?", "start": 912.182, "speaker": "SPEAKER_05"}, {"end": 920.809, "text": "And the capabilities that we have to reach people.", "start": 916.666, "speaker": "SPEAKER_05"}, {"end": 925.814, "text": "A lot of times you hear people talking about gamification of apps and things like this.", "start": 920.809, "speaker": "SPEAKER_05"}, {"end": 937.972, "text": "While there's certainly some bad things that can happen there, I think there's a lot of things that we can learn from the way Games engage people in the way they spread and the way people get into them.", "start": 925.814, "speaker": "SPEAKER_05"}, {"end": 941.673, "text": "Cuz clearly, they're doing pretty well in a mobile world.", "start": 937.972, "speaker": "SPEAKER_05"}, {"end": 947.934, "text": "Anyway, so the number of devices out there, pretty huge.", "start": 941.673, "speaker": "SPEAKER_05"}, {"end": 951.735, "text": "Audience growth and the potential for audience growth, huge.", "start": 947.934, "speaker": "SPEAKER_05"}, {"end": 956.055, "text": "Well, that's all well and good, but can we actually make any money, right?", "start": 951.735, "speaker": "SPEAKER_05"}, {"end": 958.676, "text": "Is anybody making money off of this mobile stuff?", "start": 956.055, "speaker": "SPEAKER_05"}, {"end": 961.388, "text": "And you're going to notice a common theme here.", "start": 959.506, "speaker": "SPEAKER_05"}, {"end": 965.711, "text": "Again, I'm going to rewind the clock back to 2006.", "start": 961.388, "speaker": "SPEAKER_05"}, {"end": 974.378, "text": "And in 2006, PayPal, which I'm sure you're all familiar with, their annual mobile payments volume was under a million dollars.", "start": 965.711, "speaker": "SPEAKER_05"}, {"end": 977.02, "text": "It was like $900,000, $800,000, something like that.", "start": 974.378, "speaker": "SPEAKER_05"}, {"end": 979.382, "text": "That's how much they did in 2006 in total mobile payments.", "start": 977.02, "speaker": "SPEAKER_05"}, {"end": 990.733, "text": "OK, so from this sub-million dollar number, and you'll notice my chart, by the way, is in billions.", "start": 984.127, "speaker": "SPEAKER_05"}, {"end": 992.955, "text": "Let's look at what happened over the past 10 years.", "start": 990.733, "speaker": "SPEAKER_05"}, {"end": 1003.785, "text": "The first few years were just in the millions, so they didn't even register on my chart, because last year they did $102 billion in total mobile sales.", "start": 992.955, "speaker": "SPEAKER_05"}, {"end": 1007.188, "text": "So again, we see this huge spiking chart up and to the right.", "start": 1003.785, "speaker": "SPEAKER_05"}, {"end": 1012.379, "text": "$102 billion from under $1 million in just a 10-year period.", "start": 1008.697, "speaker": "SPEAKER_05"}, {"end": 1013.52, "text": "Pretty dramatic.", "start": 1012.379, "speaker": "SPEAKER_05"}, {"end": 1019.063, "text": "And you may say, OK, well, good for PayPal, but my conversions on mobile are lower than desktop.", "start": 1013.52, "speaker": "SPEAKER_05"}, {"end": 1019.743, "text": "Anybody thinking that?", "start": 1019.063, "speaker": "SPEAKER_05"}, {"end": 1024.923, "text": "But that doesn't mean the opportunity isn't there.", "start": 1021.36, "speaker": "SPEAKER_05"}, {"end": 1032.688, "text": "It doesn't mean that all of these devices, all of these audiences doesn't create this immense opportunity for revenue that flat out didn't exist there before.", "start": 1024.923, "speaker": "SPEAKER_05"}, {"end": 1036.451, "text": "So to illustrate, let's compare the biggest online shopping days of the year.", "start": 1032.688, "speaker": "SPEAKER_05"}, {"end": 1038.493, "text": "And we'll look at the US first.", "start": 1036.451, "speaker": "SPEAKER_05"}, {"end": 1044.277, "text": "So in the US, our biggest online shopping day of the year is this thing called Cyber Monday.", "start": 1038.493, "speaker": "SPEAKER_05"}, {"end": 1051.329, "text": "And in 2006, you get the theme now.", "start": 1048.066, "speaker": "SPEAKER_05"}, {"end": 1054.232, "text": "By the way, I had to just dig for all this old data.", "start": 1051.329, "speaker": "SPEAKER_05"}, {"end": 1058.476, "text": "The internet doesn't work so well if you look for things that are older than a day.", "start": 1054.232, "speaker": "SPEAKER_05"}, {"end": 1061.999, "text": "Have you noticed that?", "start": 1058.476, "speaker": "SPEAKER_05"}, {"end": 1068.025, "text": "So back in 2006, Cyber Monday, which is the biggest online shopping day in the United States, about $610 million.", "start": 1061.999, "speaker": "SPEAKER_05"}, {"end": 1072.543, "text": "Pretty good.", "start": 1071.743, "speaker": "SPEAKER_05"}, {"end": 1078.105, "text": "And if you follow the trend line upward, you can see that 10 years later, it was 3.4 billion.", "start": 1072.543, "speaker": "SPEAKER_05"}, {"end": 1080.245, "text": "So very substantial growth.", "start": 1078.105, "speaker": "SPEAKER_05"}, {"end": 1085.247, "text": "Clearly, a lot of people buying things online and quite a big deal.", "start": 1080.245, "speaker": "SPEAKER_05"}, {"end": 1097.71, "text": "Now, when you look at the amount of sales that happened on mobile devices in the United States on the biggest shopping day of the year, so it's when most people are buying, you'll see it was about 30% of sales completed on mobile.", "start": 1085.247, "speaker": "SPEAKER_05"}, {"end": 1103.028, "text": "Not too shabby when you look at what is 30% of $3.4 billion.", "start": 1099.005, "speaker": "SPEAKER_05"}, {"end": 1108.592, "text": "It's a lot of money moving through these tiny little screens in our hands.", "start": 1103.028, "speaker": "SPEAKER_05"}, {"end": 1115.878, "text": "To look at the different side of the coin here, we're going to look at the biggest shopping day of the year in China, which was actually invented in 2009.", "start": 1108.592, "speaker": "SPEAKER_05"}, {"end": 1120.381, "text": "It's pretty awesome that you can just invent these things.", "start": 1115.878, "speaker": "SPEAKER_05"}, {"end": 1122.303, "text": "Cyber Monday was also invented.", "start": 1120.381, "speaker": "SPEAKER_05"}, {"end": 1128.421, "text": "And so Singles Day was also invented as a holiday shopping day, but in China.", "start": 1123.777, "speaker": "SPEAKER_05"}, {"end": 1131.583, "text": "In fact, the Alibaba company was really behind this.", "start": 1128.421, "speaker": "SPEAKER_05"}, {"end": 1135.006, "text": "And so when it kicked off in 2009, it was doing pretty well.", "start": 1131.583, "speaker": "SPEAKER_05"}, {"end": 1138.709, "text": "And in 2016, Singles Day, $17.8 billion within a single shopping day online.", "start": 1135.006, "speaker": "SPEAKER_05"}, {"end": 1153.574, "text": "And if you now look at how much of that happened on mobile, 84% of it happened on mobile.", "start": 1146.211, "speaker": "SPEAKER_05"}, {"end": 1162.077, "text": "So not only is China dwarfing the US in terms of cumulative sales, but they clearly got their stuff together on mobile.", "start": 1153.574, "speaker": "SPEAKER_05"}, {"end": 1167.539, "text": "And I think this illustrates the gap, which you can call this a problem.", "start": 1162.077, "speaker": "SPEAKER_05"}, {"end": 1169.12, "text": "I call this an opportunity.", "start": 1167.539, "speaker": "SPEAKER_05"}, {"end": 1170.26, "text": "Clearly.", "start": 1169.12, "speaker": "SPEAKER_05"}, {"end": 1179.497, "text": "Some parts of this world are able to get to almost 90% mobile sales on the biggest shopping day of the year.", "start": 1172.833, "speaker": "SPEAKER_05"}, {"end": 1181.558, "text": "Some of us have not gotten there yet.", "start": 1179.497, "speaker": "SPEAKER_05"}, {"end": 1187.04, "text": "But there is obviously a path to getting there.", "start": 1181.558, "speaker": "SPEAKER_05"}, {"end": 1190.962, "text": "The issue is mobile commerce just isn't evenly distributed yet.", "start": 1187.04, "speaker": "SPEAKER_05"}, {"end": 1197.586, "text": "But the potential, I think, is already evenly distributed yet, which makes it a lot easier to take advantage of this.", "start": 1190.962, "speaker": "SPEAKER_05"}, {"end": 1200.888, "text": "Because if the potential wasn't there, then it would be a much harder problem.", "start": 1197.586, "speaker": "SPEAKER_05"}, {"end": 1208.566, "text": "And getting from 30% to 84% mobile sales is a lot of opportunity for retailers.", "start": 1201.763, "speaker": "SPEAKER_05"}, {"end": 1215.328, "text": "Increasingly, in the United States, more and more of them are actually now able to start taking advantage of this because they focused on it.", "start": 1208.566, "speaker": "SPEAKER_05"}, {"end": 1222.291, "text": "So Walmart this year, which is a very big retailer, as I'm sure you know, they boasted they had 70% mobile traffic.", "start": 1215.328, "speaker": "SPEAKER_05"}, {"end": 1228.584, "text": "60% of their Black Friday is another.", "start": 1226.142, "speaker": "SPEAKER_05"}, {"end": 1229.945, "text": "I don't know why we have these weird names.", "start": 1228.584, "speaker": "SPEAKER_05"}, {"end": 1232.106, "text": "It's another big shopping day in the US.", "start": 1229.945, "speaker": "SPEAKER_05"}, {"end": 1238.872, "text": "60% of their Black Friday orders happened on mobile, which is double the US average.", "start": 1232.106, "speaker": "SPEAKER_05"}, {"end": 1242.574, "text": "So if you focus on this and actually work on it, you can take advantage of this opportunity.", "start": 1238.872, "speaker": "SPEAKER_05"}, {"end": 1249.98, "text": "Target, which is another big retailer in the US, said 60% of their Cyber Monday stuff happened on mobile.", "start": 1242.574, "speaker": "SPEAKER_05"}, {"end": 1251.802, "text": "Again, double that average.", "start": 1249.98, "speaker": "SPEAKER_05"}, {"end": 1260.259, "text": "So if you can look at the quantity of money moving and you can start to have an impact here, you can actually make a big dent.", "start": 1252.733, "speaker": "SPEAKER_05"}, {"end": 1268.747, "text": "And I like these examples because they illustrate work pays off.", "start": 1260.259, "speaker": "SPEAKER_05"}, {"end": 1273.411, "text": "A year or two earlier, this was not the numbers that Walmart and Target had.", "start": 1268.747, "speaker": "SPEAKER_05"}, {"end": 1275.913, "text": "And now, through a concentrated effort, they have them.", "start": 1273.411, "speaker": "SPEAKER_05"}, {"end": 1286.143, "text": "And getting these kinds of improvements on mobile is key, because I think the last 10 years have been very, very transformative, as hopefully I'll illustrate for you in a second.", "start": 1277.397, "speaker": "SPEAKER_05"}, {"end": 1289.085, "text": "But they've really just set us up for what's next.", "start": 1286.143, "speaker": "SPEAKER_05"}, {"end": 1291.927, "text": "And so in order to do that, let me look at this chart real quick.", "start": 1289.085, "speaker": "SPEAKER_05"}, {"end": 1293.828, "text": "So this is cumulative PCs now.", "start": 1291.927, "speaker": "SPEAKER_05"}, {"end": 1297.85, "text": "This isn't yearly sales that we're looking at before.", "start": 1293.828, "speaker": "SPEAKER_05"}, {"end": 1298.691, "text": "This is cumulative.", "start": 1297.85, "speaker": "SPEAKER_05"}, {"end": 1306.096, "text": "How many is there in operations on planet Earth, the rock spinning around the sun today?", "start": 1298.691, "speaker": "SPEAKER_05"}, {"end": 1311.787, "text": "how many cumulative smartphones are out there in operation.", "start": 1307.924, "speaker": "SPEAKER_05"}, {"end": 1315.149, "text": "And the cumulative smartphone number is approaching 3 billion.", "start": 1311.787, "speaker": "SPEAKER_05"}, {"end": 1320.112, "text": "It's probably right at 3 billion now, because these cumulative stats I couldn't pull for the end of last year.", "start": 1315.149, "speaker": "SPEAKER_05"}, {"end": 1321.133, "text": "But it's about 3 billion.", "start": 1320.112, "speaker": "SPEAKER_05"}, {"end": 1327.498, "text": "So PCs are about $1.2, $1.3 billion or something like that.", "start": 1322.415, "speaker": "SPEAKER_05"}, {"end": 1330.339, "text": "Mobile phones, these are non-smartphones.", "start": 1327.498, "speaker": "SPEAKER_05"}, {"end": 1333.681, "text": "If you look at mobile phones, it's approaching around $5 billion.", "start": 1330.339, "speaker": "SPEAKER_05"}, {"end": 1341.145, "text": "And this number is actually kind of flattening a touch, because what's happening is mobile phones are effectively becoming smartphones.", "start": 1333.681, "speaker": "SPEAKER_05"}, {"end": 1347.368, "text": "And so you can expect this line to effectively get replaced by this line over time.", "start": 1341.145, "speaker": "SPEAKER_05"}, {"end": 1349.509, "text": "So the mobile line becomes the smartphone line.", "start": 1347.368, "speaker": "SPEAKER_05"}, {"end": 1353.589, "text": "Next line, I think, is the most interesting.", "start": 1351.927, "speaker": "SPEAKER_05"}, {"end": 1356.712, "text": "These are the number of adults on the planet.", "start": 1353.589, "speaker": "SPEAKER_05"}, {"end": 1363.079, "text": "And we've got about 5.5 billion out of 7-something billion people on the rock.", "start": 1356.712, "speaker": "SPEAKER_05"}, {"end": 1371.707, "text": "And I'm going to draw a line here at this point that I kept talking about for 2006, because this illustrates what has happened in the past 10 years.", "start": 1363.079, "speaker": "SPEAKER_05"}, {"end": 1380.545, "text": "And one, the first thing to notice, we're talking about global scale here.", "start": 1375.981, "speaker": "SPEAKER_05"}, {"end": 1385.449, "text": "We're literally talking about something that touches all the human beings on the planet.", "start": 1380.545, "speaker": "SPEAKER_05"}, {"end": 1396.378, "text": "And as smartphones turn over mobile phones, and as that line starts to approach the number of adults on the planet, what are we going to do about that?", "start": 1385.449, "speaker": "SPEAKER_05"}, {"end": 1403.244, "text": "What happens when all adults on our planet are now connected through networked supercomputers in their pockets?", "start": 1396.378, "speaker": "SPEAKER_05"}, {"end": 1415.32, "text": "Things filled with sensors, things filled with processing power, things connected to the cloud, able to access all kinds of data, collect all kinds of information.", "start": 1405.753, "speaker": "SPEAKER_05"}, {"end": 1418.183, "text": "I think it really changes the dynamics.", "start": 1415.32, "speaker": "SPEAKER_05"}, {"end": 1421.805, "text": "And this is why I say we're entering this world of deployment.", "start": 1418.183, "speaker": "SPEAKER_05"}, {"end": 1424.808, "text": "These trends are pretty set now.", "start": 1421.805, "speaker": "SPEAKER_05"}, {"end": 1428.671, "text": "It's pretty clear what's going to happen, unless, who knows, right?", "start": 1424.808, "speaker": "SPEAKER_05"}, {"end": 1429.932, "text": "I can't predict the future.", "start": 1428.671, "speaker": "SPEAKER_05"}, {"end": 1437.483, "text": "Hopefully, looking at a chart like this, you're convinced of the mobile opportunity.", "start": 1432.081, "speaker": "SPEAKER_05"}, {"end": 1443.125, "text": "So huge growth in devices equals big audiences equals big potential for revenue.", "start": 1437.483, "speaker": "SPEAKER_05"}, {"end": 1447.406, "text": "And there's clearly people taking advantage of that revenue opportunity in huge ways.", "start": 1443.125, "speaker": "SPEAKER_05"}, {"end": 1454.334, "text": "But what have we actually learned over the past 10 years about making things for mobile?", "start": 1448.447, "speaker": "SPEAKER_05"}, {"end": 1462.925, "text": "How has our understanding of mobile design changed as we've gone up these lines and we've seen these curves grow in this manner?", "start": 1454.334, "speaker": "SPEAKER_05"}, {"end": 1467.082, "text": "And that's what I really want to spend most of our time talking about.", "start": 1464.18, "speaker": "SPEAKER_05"}, {"end": 1472.206, "text": "The first part here is just set up to convince you this stuff matters in case you're not already convinced.", "start": 1467.082, "speaker": "SPEAKER_05"}, {"end": 1473.667, "text": "I hope you're already convinced.", "start": 1472.206, "speaker": "SPEAKER_05"}, {"end": 1480.152, "text": "But a lot of times, seeing it from this kind of perspective, I think, helps hammer the point home.", "start": 1473.667, "speaker": "SPEAKER_05"}, {"end": 1485.456, "text": "And even though we've got three hours together, I can't possibly cover everything we've learned in 10 years.", "start": 1480.152, "speaker": "SPEAKER_05"}, {"end": 1488.698, "text": "So I'm just going to talk about a couple of things that are interesting to me.", "start": 1485.456, "speaker": "SPEAKER_05"}, {"end": 1488.919, "text": "Sorry.", "start": 1488.698, "speaker": "SPEAKER_05"}, {"end": 1496.79, "text": "And in particular, what we'll talk about is sort of the current state of optimizing for mobile, what's possible today.", "start": 1490.805, "speaker": "SPEAKER_05"}, {"end": 1503.976, "text": "But most importantly, I'm going to look at some kind of weird examples or some things on the fringes that you may say, huh, why are we looking at that?", "start": 1496.79, "speaker": "SPEAKER_05"}, {"end": 1508.56, "text": "But what I want that to do is to inspire us for what's next.", "start": 1503.976, "speaker": "SPEAKER_05"}, {"end": 1511.442, "text": "I want us to look forward the next 10 years.", "start": 1508.56, "speaker": "SPEAKER_05"}, {"end": 1516.887, "text": "Because frankly, what happened for a large portion of this past 10 years is we were too caught up in the past.", "start": 1511.442, "speaker": "SPEAKER_05"}, {"end": 1522.269, "text": "We're still thinking of everything from the desktop perspective, and you still hear this.", "start": 1517.868, "speaker": "SPEAKER_05"}, {"end": 1527.63, "text": "Even today, people are like, well, my desktop conversions are higher than my mobile conversions.", "start": 1522.269, "speaker": "SPEAKER_05"}, {"end": 1538.072, "text": "We're just thinking in terms of what happened before, as opposed to looking today at the opportunity, and more importantly, looking in the future at the opportunity.", "start": 1527.63, "speaker": "SPEAKER_05"}, {"end": 1549.831, "text": "That's how you actually take advantage of what's going to happen, by being prepared for it, by thinking ahead, by looking toward the future, hence kind of the the purpose of this talk.", "start": 1538.072, "speaker": "SPEAKER_05"}, {"end": 1556.013, "text": "So 10 years ago, when the first iPhone was announced, things were pretty darn simple, right?", "start": 1549.831, "speaker": "SPEAKER_05"}, {"end": 1563.675, "text": "There was a camera that allowed you to take a picture, and then you could look at that picture.", "start": 1556.013, "speaker": "SPEAKER_05"}, {"end": 1565.976, "text": "That was it.", "start": 1563.675, "speaker": "SPEAKER_05"}, {"end": 1569.757, "text": "Seems pretty crappy, right?", "start": 1565.976, "speaker": "SPEAKER_05"}, {"end": 1570.237, "text": "Oh, right.", "start": 1569.757, "speaker": "SPEAKER_05"}, {"end": 1572.118, "text": "I didn't take a picture.", "start": 1570.237, "speaker": "SPEAKER_05"}, {"end": 1572.218, "text": "OK.", "start": 1572.118, "speaker": "SPEAKER_05"}, {"end": 1575.596, "text": "It doesn't seem very compelling.", "start": 1573.875, "speaker": "SPEAKER_05"}, {"end": 1584.282, "text": "But over the years, each subsequent version of the iPhone and of iOS added new features and capabilities.", "start": 1575.596, "speaker": "SPEAKER_05"}, {"end": 1587.844, "text": "So when we started from that first version, we could take a picture, we could view it.", "start": 1584.282, "speaker": "SPEAKER_05"}, {"end": 1590.106, "text": "Then all of a sudden, we were able to do video.", "start": 1587.844, "speaker": "SPEAKER_05"}, {"end": 1594.389, "text": "Then we were able to flip the camera around because there's two, so enter selfies.", "start": 1590.106, "speaker": "SPEAKER_05"}, {"end": 1597.669, "text": "You could turn a flash on and off.", "start": 1595.907, "speaker": "SPEAKER_05"}, {"end": 1599.491, "text": "There was now grids.", "start": 1597.669, "speaker": "SPEAKER_05"}, {"end": 1600.892, "text": "You could have HDR.", "start": 1599.491, "speaker": "SPEAKER_05"}, {"end": 1609.561, "text": "You could take photos with square boxes, right, kind of stemming from the Instagram explosion back when we got to iOS 7.", "start": 1600.892, "speaker": "SPEAKER_05"}, {"end": 1615.847, "text": "You can sort of note the design change that happened here when we moved to flat design.", "start": 1609.561, "speaker": "SPEAKER_05"}, {"end": 1617.128, "text": "Anybody remember flat design?", "start": 1615.847, "speaker": "SPEAKER_05"}, {"end": 1618.778, "text": "No?", "start": 1618.438, "speaker": "SPEAKER_05"}, {"end": 1622.8, "text": "It used to be the only thing anybody ever talked about in the design space a while ago.", "start": 1618.778, "speaker": "SPEAKER_05"}, {"end": 1632.005, "text": "I have to show you my favorite flat design image just because, I don't know, I want to.", "start": 1622.8, "speaker": "SPEAKER_05"}, {"end": 1634.086, "text": "Where did I put my favorite flat design image?", "start": 1632.005, "speaker": "SPEAKER_05"}, {"end": 1636.687, "text": "OK, well, this will take me half a second.", "start": 1635.246, "speaker": "SPEAKER_05"}, {"end": 1638.968, "text": "But trust me, it's totally worth it.", "start": 1636.687, "speaker": "SPEAKER_05"}, {"end": 1644.43, "text": "It's worth waiting for me to wade through these slides and find my flat design image.", "start": 1638.968, "speaker": "SPEAKER_05"}, {"end": 1648.892, "text": "Oh, it is going to elude me.", "start": 1644.43, "speaker": "SPEAKER_05"}, {"end": 1650.733, "text": "Oh, come on.", "start": 1648.892, "speaker": "SPEAKER_05"}, {"end": 1658.956, "text": "All right, one more.", "start": 1650.733, "speaker": "SPEAKER_05"}, {"end": 1660.597, "text": "And I promise I'll stop and give up.", "start": 1658.956, "speaker": "SPEAKER_05"}, {"end": 1666.762, "text": "OK, yes, yes.", "start": 1663.74, "speaker": "SPEAKER_05"}, {"end": 1669.984, "text": "Window.", "start": 1666.762, "speaker": "SPEAKER_05"}, {"end": 1671.305, "text": "All right, I'm losing you, aren't I?", "start": 1669.984, "speaker": "SPEAKER_05"}, {"end": 1676.889, "text": "I swear, this joke is worth it.", "start": 1671.305, "speaker": "SPEAKER_05"}, {"end": 1678.11, "text": "Just believe me.", "start": 1676.889, "speaker": "SPEAKER_05"}, {"end": 1681.632, "text": "Oh, come on.", "start": 1678.11, "speaker": "SPEAKER_05"}, {"end": 1685.255, "text": "I wish somebody could invent something that could take the picture that's in my head.", "start": 1681.632, "speaker": "SPEAKER_05"}, {"end": 1689.436, "text": "and basically surface it on my screen?", "start": 1686.355, "speaker": "SPEAKER_05"}, {"end": 1692.298, "text": "Is there something like that that exists?", "start": 1689.436, "speaker": "SPEAKER_05"}, {"end": 1695.339, "text": "I think Google's working on it.", "start": 1692.298, "speaker": "SPEAKER_05"}, {"end": 1696.94, "text": "OK, one last second.", "start": 1695.339, "speaker": "SPEAKER_05"}, {"end": 1705.664, "text": "And if I don't find it, I will give up, admit defeat, and you can chastise me.", "start": 1696.94, "speaker": "SPEAKER_05"}, {"end": 1712.106, "text": "Ah, yes.", "start": 1705.664, "speaker": "SPEAKER_05"}, {"end": 1713.327, "text": "Beware flat design.", "start": 1712.106, "speaker": "SPEAKER_05"}, {"end": 1725.232, "text": "Original Hershey's logo, flat design version, piece of poo.", "start": 1715.688, "speaker": "SPEAKER_05"}, {"end": 1728.994, "text": "That was worth waiting for, right?", "start": 1725.232, "speaker": "SPEAKER_05"}, {"end": 1733.756, "text": "Please tell me, else I'll feel really sad later.", "start": 1728.994, "speaker": "SPEAKER_05"}, {"end": 1737.798, "text": "So yeah, iOS 7 did this flat design thing that everybody freaked out about.", "start": 1733.756, "speaker": "SPEAKER_05"}, {"end": 1740.866, "text": "Here we go.", "start": 1740.286, "speaker": "SPEAKER_05"}, {"end": 1746.467, "text": "So we added selfies, we added flash, we added HDR, square photos.", "start": 1740.866, "speaker": "SPEAKER_05"}, {"end": 1749.748, "text": "Of course, it doesn't stop here, right?", "start": 1746.467, "speaker": "SPEAKER_05"}, {"end": 1765.652, "text": "Continuing down the trend, we now have slow motion video, time lapse video, panoramas, timers, live photos, portrait photos, telephotos, filtering, you name it.", "start": 1749.748, "speaker": "SPEAKER_05"}, {"end": 1777.74, "text": "So we've gone from this very simple just take a picture and look at it to, wow, look at all the stuff you can do with the camera in your pocket.", "start": 1767.975, "speaker": "SPEAKER_05"}, {"end": 1784.604, "text": "And when we look at this collectively, I work all night on the animations for you people.", "start": 1777.74, "speaker": "SPEAKER_05"}, {"end": 1785.745, "text": "I should actually say that.", "start": 1784.604, "speaker": "SPEAKER_05"}, {"end": 1786.705, "text": "Let me just say that.", "start": 1785.745, "speaker": "SPEAKER_05"}, {"end": 1789.987, "text": "These animations that you see, all this stuff, all night.", "start": 1786.705, "speaker": "SPEAKER_05"}, {"end": 1793.769, "text": "That's all I do.", "start": 1789.987, "speaker": "SPEAKER_05"}, {"end": 1797.631, "text": "So some of you may look at this animation, which did I mention I worked all night on?", "start": 1793.769, "speaker": "SPEAKER_05"}, {"end": 1805.369, "text": "You may look at it and say, oh, this is a classic case of feature creep, right?", "start": 1799.065, "speaker": "SPEAKER_05"}, {"end": 1808.211, "text": "I longed for the days that it was easy to use this stuff.", "start": 1805.369, "speaker": "SPEAKER_05"}, {"end": 1810.633, "text": "Now there's so many things I can't use it.", "start": 1808.211, "speaker": "SPEAKER_05"}, {"end": 1815.248, "text": "A very common reaction to seeing this, people say that, right?", "start": 1811.744, "speaker": "SPEAKER_05"}, {"end": 1818.192, "text": "It used to be such a simple thing, now it's gotten so complicated.", "start": 1815.248, "speaker": "SPEAKER_05"}, {"end": 1823.558, "text": "And looking at this picture, it clearly feels like, holy cow, tons of features have crammed in here.", "start": 1818.192, "speaker": "SPEAKER_05"}, {"end": 1829.365, "text": "Other people look at this process and they say, ah, this is the perfect example of building a minimum viable product.", "start": 1823.558, "speaker": "SPEAKER_05"}, {"end": 1833.815, "text": "You basically build the first version, and people like it.", "start": 1830.392, "speaker": "SPEAKER_05"}, {"end": 1834.375, "text": "OK, cool.", "start": 1833.815, "speaker": "SPEAKER_05"}, {"end": 1836.817, "text": "Then you start investing more in it.", "start": 1834.375, "speaker": "SPEAKER_05"}, {"end": 1838.218, "text": "They're happy with the skateboard.", "start": 1836.817, "speaker": "SPEAKER_05"}, {"end": 1839.639, "text": "They get happy with the scooter.", "start": 1838.218, "speaker": "SPEAKER_05"}, {"end": 1842.342, "text": "They're happy with the bike, happy with the motorcycle, and happy with the car.", "start": 1839.639, "speaker": "SPEAKER_05"}, {"end": 1846.265, "text": "They're just happy people all the way around.", "start": 1842.342, "speaker": "SPEAKER_05"}, {"end": 1855.072, "text": "And the purpose of this MVP sort of mindset is that you don't spend way too much time adding a pile of features onto something that nobody wants anyway.", "start": 1846.265, "speaker": "SPEAKER_05"}, {"end": 1857.694, "text": "So you start with something, figure out if they like it.", "start": 1855.072, "speaker": "SPEAKER_05"}, {"end": 1859.155, "text": "Then over time, you build it up.", "start": 1857.694, "speaker": "SPEAKER_05"}, {"end": 1867.171, "text": "And I'm just kind of curious, having heard arguments on both sides, which is it?", "start": 1860.186, "speaker": "SPEAKER_05"}, {"end": 1868.672, "text": "Classic case of feature creep?", "start": 1867.171, "speaker": "SPEAKER_05"}, {"end": 1872.194, "text": "Who says this is feature creep?", "start": 1868.672, "speaker": "SPEAKER_05"}, {"end": 1873.635, "text": "Wow, just three people?", "start": 1872.194, "speaker": "SPEAKER_05"}, {"end": 1876.457, "text": "Okay, interesting.", "start": 1873.635, "speaker": "SPEAKER_05"}, {"end": 1881.82, "text": "Or I guess the rest of you say this is MVP kind of thing at scale?", "start": 1876.457, "speaker": "SPEAKER_05"}, {"end": 1883.902, "text": "Don't be afraid.", "start": 1881.82, "speaker": "SPEAKER_05"}, {"end": 1888.905, "text": "Okay, so seven of you, about 93% do not like to raise their hand in little surveys.", "start": 1883.902, "speaker": "SPEAKER_05"}, {"end": 1895.154, "text": "I'm going to call that the margin of error here for everything that we do.", "start": 1890.592, "speaker": "SPEAKER_05"}, {"end": 1896.494, "text": "I just think it's an interesting question.", "start": 1895.154, "speaker": "SPEAKER_05"}, {"end": 1901.576, "text": "People will debate both sides of it and have very strong opinions on both sides.", "start": 1896.494, "speaker": "SPEAKER_05"}, {"end": 1906.478, "text": "So of course, the camera isn't the only feature of smartphones to have gained popularity over the years.", "start": 1901.576, "speaker": "SPEAKER_05"}, {"end": 1909.799, "text": "Does anybody know about this thing called messaging?", "start": 1906.478, "speaker": "SPEAKER_05"}, {"end": 1911.519, "text": "Of course you do.", "start": 1909.799, "speaker": "SPEAKER_05"}, {"end": 1919.262, "text": "And if we look at the past 10 years of messaging, again, 10 years ago, you were able to send a text message.", "start": 1911.519, "speaker": "SPEAKER_05"}, {"end": 1922.142, "text": "I know.", "start": 1921.762, "speaker": "SPEAKER_05"}, {"end": 1923.463, "text": "I know.", "start": 1922.142, "speaker": "SPEAKER_05"}, {"end": 1925.805, "text": "It just doesn't compare, does it?", "start": 1923.463, "speaker": "SPEAKER_05"}, {"end": 1929.107, "text": "Looking at the past 10 years of messaging, we see a similar pattern.", "start": 1925.805, "speaker": "SPEAKER_05"}, {"end": 1936.051, "text": "We go from this very simple thing to now you can, in addition to sending text, you can hit this little button.", "start": 1929.107, "speaker": "SPEAKER_05"}, {"end": 1938.032, "text": "You can go take a selfie instantly.", "start": 1936.051, "speaker": "SPEAKER_05"}, {"end": 1939.333, "text": "You can flip it over.", "start": 1938.032, "speaker": "SPEAKER_05"}, {"end": 1940.313, "text": "You can do recording.", "start": 1939.333, "speaker": "SPEAKER_05"}, {"end": 1942.575, "text": "You can send anything from your camera roll.", "start": 1940.313, "speaker": "SPEAKER_05"}, {"end": 1944.336, "text": "That's pretty useful.", "start": 1942.575, "speaker": "SPEAKER_05"}, {"end": 1945.517, "text": "You can do searches.", "start": 1944.336, "speaker": "SPEAKER_05"}, {"end": 1947.138, "text": "You can go find animated GIFs.", "start": 1945.517, "speaker": "SPEAKER_05"}, {"end": 1948.979, "text": "You can record audio.", "start": 1947.138, "speaker": "SPEAKER_05"}, {"end": 1950.42, "text": "Send whatever you're saying.", "start": 1948.979, "speaker": "SPEAKER_05"}, {"end": 1954.69, "text": "I mean, you can literally, this is, I had no idea this thing even existed.", "start": 1951.348, "speaker": "SPEAKER_05"}, {"end": 1957.091, "text": "If you like fart around on here, God knows what you'll find.", "start": 1954.69, "speaker": "SPEAKER_05"}, {"end": 1960.874, "text": "You can record a video and draw over it with like fireworky actions.", "start": 1957.091, "speaker": "SPEAKER_05"}, {"end": 1968.418, "text": "All right, so you're recording a video and you can like light your head on fire and send that to somebody.", "start": 1960.874, "speaker": "SPEAKER_05"}, {"end": 1971.52, "text": "I guess there's a time and place for that.", "start": 1968.418, "speaker": "SPEAKER_05"}, {"end": 1972.881, "text": "Perhaps now, right?", "start": 1971.52, "speaker": "SPEAKER_05"}, {"end": 1974.462, "text": "Oh my God, have you seen his animations?", "start": 1972.881, "speaker": "SPEAKER_05"}, {"end": 1983.048, "text": "Not only that, there's a whole app store for things to add to messaging.", "start": 1978.007, "speaker": "SPEAKER_05"}, {"end": 1993.17, "text": "There's a whole category of software that you can insert into messaging to do all kinds of things, to send pretty weddings or wedding doodles to each other.", "start": 1983.048, "speaker": "SPEAKER_05"}, {"end": 2000.151, "text": "I mean, there's a whole wedding category of messaging apps.", "start": 1993.17, "speaker": "SPEAKER_05"}, {"end": 2002.032, "text": "Same story with how we share.", "start": 2000.151, "speaker": "SPEAKER_05"}, {"end": 2006.673, "text": "Quite literally, back in 2006, you know how you share one of those photos you took of the camera?", "start": 2002.032, "speaker": "SPEAKER_05"}, {"end": 2007.413, "text": "You email it.", "start": 2006.673, "speaker": "SPEAKER_05"}, {"end": 2013.1, "text": "You would email the photo to another human being.", "start": 2010.257, "speaker": "SPEAKER_05"}, {"end": 2017.024, "text": "That was how you took the picture from your phone and gave it to another human being.", "start": 2013.1, "speaker": "SPEAKER_05"}, {"end": 2020.156, "text": "And where are we at today?", "start": 2018.176, "speaker": "SPEAKER_05"}, {"end": 2034.7, "text": "Well, you have this whole share sheet thing, and inside of there, you can mail, message, tweet, save, hang out, PDF, cloud share, Slackify, LinkedInify, Flickrify, and that's not all.", "start": 2020.156, "speaker": "SPEAKER_05"}, {"end": 2043.442, "text": "Then you can go into the next layer of options, and you can print, and airplay, and assign to a contact, and put in the cloud, and God knows what else.", "start": 2034.7, "speaker": "SPEAKER_05"}, {"end": 2049.55, "text": "It's almost a dizzying array of options of what you can do with that photo.", "start": 2045.367, "speaker": "SPEAKER_05"}, {"end": 2052.451, "text": "I should note, Apple tries to manage some of this complexity.", "start": 2049.55, "speaker": "SPEAKER_05"}, {"end": 2063.878, "text": "So at first, you get this top-level stuff, and then if you go through all this, you can get to more things, and then only if you go even deeper, they try to stagger some of this complexity.", "start": 2052.451, "speaker": "SPEAKER_05"}, {"end": 2067.68, "text": "But the capabilities are huge.", "start": 2063.878, "speaker": "SPEAKER_05"}, {"end": 2072.583, "text": "Of course, this introduces discovery versus usage problems, and on and on we can talk about that.", "start": 2067.68, "speaker": "SPEAKER_05"}, {"end": 2081.232, "text": "But let's move away from Apple for a minute and look at the same iterative process within mobile app design.", "start": 2074.09, "speaker": "SPEAKER_05"}, {"end": 2086.034, "text": "So this is the eBay Android app roughly 10 years ago.", "start": 2081.232, "speaker": "SPEAKER_05"}, {"end": 2089.595, "text": "Looks awesome, doesn't it?", "start": 2086.034, "speaker": "SPEAKER_05"}, {"end": 2093.256, "text": "And here's an iteration, the next iteration of it.", "start": 2089.595, "speaker": "SPEAKER_05"}, {"end": 2097.017, "text": "You can see that they added some things, daily deals, messages.", "start": 2093.256, "speaker": "SPEAKER_05"}, {"end": 2098.678, "text": "And next iteration.", "start": 2097.017, "speaker": "SPEAKER_05"}, {"end": 2102.199, "text": "Again, I've added a couple things.", "start": 2100.017, "speaker": "SPEAKER_05"}, {"end": 2106.422, "text": "Right about here is where I think they bought Red Laser, that barcode scanning app.", "start": 2102.199, "speaker": "SPEAKER_05"}, {"end": 2109.605, "text": "Remember when barcode scanning was going to be the huge thing?", "start": 2106.422, "speaker": "SPEAKER_05"}, {"end": 2115.589, "text": "Of course, they put the little barcode scanning icon up there, and nobody knew what it was, so then they put a red line through it.", "start": 2109.605, "speaker": "SPEAKER_05"}, {"end": 2119.953, "text": "And of course, nobody knew what it was, so then they put the word scan next to it.", "start": 2115.589, "speaker": "SPEAKER_05"}, {"end": 2124.914, "text": "And probably nobody used it.", "start": 2122.352, "speaker": "SPEAKER_05"}, {"end": 2127.275, "text": "So it stayed there for a little while.", "start": 2124.914, "speaker": "SPEAKER_05"}, {"end": 2130.777, "text": "And then I think after like version five or six, it went away.", "start": 2127.275, "speaker": "SPEAKER_05"}, {"end": 2134.259, "text": "And then they kind of moved to search.", "start": 2130.777, "speaker": "SPEAKER_05"}, {"end": 2137.101, "text": "But the way they did search here, nobody could see search.", "start": 2134.259, "speaker": "SPEAKER_05"}, {"end": 2138.902, "text": "So that was probably a bit of a problem.", "start": 2137.101, "speaker": "SPEAKER_05"}, {"end": 2142.224, "text": "And in the next version, you can see that they start to realize nobody could use search.", "start": 2138.902, "speaker": "SPEAKER_05"}, {"end": 2143.345, "text": "So they just hid it outright.", "start": 2142.224, "speaker": "SPEAKER_05"}, {"end": 2145.506, "text": "Then they made it very prominent.", "start": 2143.345, "speaker": "SPEAKER_05"}, {"end": 2147.487, "text": "And then I don't even know what they're doing here.", "start": 2145.506, "speaker": "SPEAKER_05"}, {"end": 2148.468, "text": "Search is gone again.", "start": 2147.487, "speaker": "SPEAKER_05"}, {"end": 2155.544, "text": "Here they added deal frenzies, which is awesome, I guess.", "start": 2151.482, "speaker": "SPEAKER_05"}, {"end": 2159.326, "text": "And now search is back.", "start": 2155.544, "speaker": "SPEAKER_05"}, {"end": 2161.647, "text": "And da, da, da, da, da.", "start": 2159.326, "speaker": "SPEAKER_05"}, {"end": 2164.448, "text": "That release, I think, was just to tell you that PayPal separated.", "start": 2161.647, "speaker": "SPEAKER_05"}, {"end": 2165.949, "text": "On and on and on it goes.", "start": 2164.448, "speaker": "SPEAKER_05"}, {"end": 2167.97, "text": "And da, da, da, da, da, da.", "start": 2165.949, "speaker": "SPEAKER_05"}, {"end": 2173.352, "text": "All night.", "start": 2167.97, "speaker": "SPEAKER_05"}, {"end": 2176.034, "text": "So let's look at this progression all at once.", "start": 2173.352, "speaker": "SPEAKER_05"}, {"end": 2178.435, "text": "Over the past 10 years, what have we been doing?", "start": 2176.034, "speaker": "SPEAKER_05"}, {"end": 2182.588, "text": "Are we just moving pixels around?", "start": 2181.027, "speaker": "SPEAKER_05"}, {"end": 2187.409, "text": "Are we making progress?", "start": 2182.588, "speaker": "SPEAKER_05"}, {"end": 2189.47, "text": "Are we learning things over the past 10 years?", "start": 2187.409, "speaker": "SPEAKER_05"}, {"end": 2193.771, "text": "Or are we just putting search here, putting search there, putting search here, putting search there?", "start": 2189.47, "speaker": "SPEAKER_05"}, {"end": 2194.432, "text": "Should we have search?", "start": 2193.771, "speaker": "SPEAKER_05"}, {"end": 2194.872, "text": "I don't know.", "start": 2194.432, "speaker": "SPEAKER_05"}, {"end": 2195.712, "text": "Let's try without search.", "start": 2194.872, "speaker": "SPEAKER_05"}, {"end": 2196.272, "text": "OK.", "start": 2195.712, "speaker": "SPEAKER_05"}, {"end": 2196.832, "text": "Put it here.", "start": 2196.272, "speaker": "SPEAKER_05"}, {"end": 2197.333, "text": "Put it there.", "start": 2196.832, "speaker": "SPEAKER_05"}, {"end": 2197.853, "text": "Put it there.", "start": 2197.333, "speaker": "SPEAKER_05"}, {"end": 2198.313, "text": "Put it there.", "start": 2197.853, "speaker": "SPEAKER_05"}, {"end": 2207.741, "text": "I ask this because this reminds me a lot of what happened with graphical user interfaces.", "start": 2202.316, "speaker": "SPEAKER_05"}, {"end": 2211.145, "text": "Graphical user interfaces have been around a lot longer than mobile, right?", "start": 2207.741, "speaker": "SPEAKER_05"}, {"end": 2213.207, "text": "Over 30 years.", "start": 2211.145, "speaker": "SPEAKER_05"}, {"end": 2216.67, "text": "In fact, let's look at the progression of graphical user interfaces for a second.", "start": 2213.207, "speaker": "SPEAKER_05"}, {"end": 2220.193, "text": "Here's 30 years of Adobe Illustrator designed for the desktop.", "start": 2216.67, "speaker": "SPEAKER_05"}, {"end": 2224.417, "text": "30 years!", "start": 2220.193, "speaker": "SPEAKER_05"}, {"end": 2225.238, "text": "Are we making progress?", "start": 2224.417, "speaker": "SPEAKER_05"}, {"end": 2230.925, "text": "Is this good?", "start": 2230.244, "speaker": "SPEAKER_05"}, {"end": 2232.266, "text": "Did we nail it the first time?", "start": 2230.925, "speaker": "SPEAKER_05"}, {"end": 2233.206, "text": "Nailed it.", "start": 2232.266, "speaker": "SPEAKER_05"}, {"end": 2239.111, "text": "I'm just going to move icons around for the next 30 years.", "start": 2233.206, "speaker": "SPEAKER_05"}, {"end": 2240.492, "text": "First time was perfect.", "start": 2239.111, "speaker": "SPEAKER_05"}, {"end": 2245.176, "text": "Remember 87?", "start": 2240.492, "speaker": "SPEAKER_05"}, {"end": 2252.442, "text": "That's when we nailed it.", "start": 2245.176, "speaker": "SPEAKER_05"}, {"end": 2256.225, "text": "Are we doing the same thing on mobile that we did for graphical user interfaces?", "start": 2252.442, "speaker": "SPEAKER_05"}, {"end": 2258.747, "text": "Is this the path that makes sense?", "start": 2256.225, "speaker": "SPEAKER_05"}, {"end": 2267.41, "text": "Are we just applying the same processes and results from desktop design to mobile design?", "start": 2261.568, "speaker": "SPEAKER_05"}, {"end": 2276.512, "text": "Or are we taking all the things, like the iterations of moving the pixels around time and time again that we did on the desktop, are we just now, hey, let's do it on mobile.", "start": 2267.41, "speaker": "SPEAKER_05"}, {"end": 2280.834, "text": "All right, now we've been doing it for 10 years on mobile.", "start": 2276.512, "speaker": "SPEAKER_05"}, {"end": 2282.814, "text": "Is this where we should end up?", "start": 2280.834, "speaker": "SPEAKER_05"}, {"end": 2284.535, "text": "Or is there really an opportunity to do more?", "start": 2282.814, "speaker": "SPEAKER_05"}, {"end": 2290.18, "text": "And so this is kind of what I want to explore here.", "start": 2287.778, "speaker": "SPEAKER_05"}, {"end": 2297.104, "text": "And this is why I think the inspiration piece and the looking out there piece that I'm going to get into a little bit is so important.", "start": 2290.18, "speaker": "SPEAKER_05"}, {"end": 2300.106, "text": "Because me personally, I feel like we can do more.", "start": 2297.104, "speaker": "SPEAKER_05"}, {"end": 2311.594, "text": "I don't think the same 30 years of moving pixels around is the opportunity that mobile provides, especially at the scale that mobile brings.", "start": 2300.106, "speaker": "SPEAKER_05"}, {"end": 2315.296, "text": "Sure, we'll have a big impact if we bring some of this stuff to 5 billion people or so.", "start": 2311.594, "speaker": "SPEAKER_05"}, {"end": 2321.201, "text": "But how can we create new experiences?", "start": 2318.899, "speaker": "SPEAKER_05"}, {"end": 2324.383, "text": "How can we really take advantage of this opportunity?", "start": 2321.201, "speaker": "SPEAKER_05"}, {"end": 2326.584, "text": "And I think there's nothing wrong with iteration.", "start": 2324.383, "speaker": "SPEAKER_05"}, {"end": 2328.365, "text": "I actually am a huge fan of iteration.", "start": 2326.584, "speaker": "SPEAKER_05"}, {"end": 2333.589, "text": "It makes a ton of sense, as hopefully I'll illustrate going forward here.", "start": 2328.365, "speaker": "SPEAKER_05"}, {"end": 2338.472, "text": "But it depends on where we're iterating to.", "start": 2333.589, "speaker": "SPEAKER_05"}, {"end": 2345.677, "text": "This is, to me, a lot of times the missing piece when people just get into this phase of ongoing iteration and testing.", "start": 2338.472, "speaker": "SPEAKER_05"}, {"end": 2347.338, "text": "Where are you going?", "start": 2345.677, "speaker": "SPEAKER_05"}, {"end": 2347.979, "text": "What's the vision?", "start": 2347.338, "speaker": "SPEAKER_05"}, {"end": 2351.669, "text": "What are we trying to achieve?", "start": 2350.148, "speaker": "SPEAKER_05"}, {"end": 2355.612, "text": "And so to illustrate this, let's look at an example.", "start": 2351.669, "speaker": "SPEAKER_05"}, {"end": 2358.314, "text": "I want to thank Craig yesterday for bringing this one up.", "start": 2355.612, "speaker": "SPEAKER_05"}, {"end": 2362.037, "text": "I didn't know he was going to do that, but it's awesome that he did.", "start": 2358.314, "speaker": "SPEAKER_05"}, {"end": 2368.122, "text": "He outlined some of the things that I looked at over a couple of years around login.", "start": 2362.037, "speaker": "SPEAKER_05"}, {"end": 2374.767, "text": "And so I kind of want to bring us up to speed around what's happening with sign-in and use it to illustrate this example of what I'm talking about.", "start": 2368.122, "speaker": "SPEAKER_05"}, {"end": 2377.81, "text": "Now, you're all probably way too familiar with login screens, right?", "start": 2374.767, "speaker": "SPEAKER_05"}, {"end": 2380.724, "text": "Everybody seen one of these?", "start": 2379.042, "speaker": "SPEAKER_05"}, {"end": 2389.954, "text": "Some studies say people see 15 of these a day, which I'm sorry, not what I intended when I got into software design.", "start": 2380.724, "speaker": "SPEAKER_05"}, {"end": 2392.056, "text": "But you may say, well, what's wrong with this?", "start": 2389.954, "speaker": "SPEAKER_05"}, {"end": 2394.739, "text": "Everybody knows login screen, username, password, right?", "start": 2392.056, "speaker": "SPEAKER_05"}, {"end": 2395.34, "text": "Everybody does it.", "start": 2394.739, "speaker": "SPEAKER_05"}, {"end": 2397.634, "text": "Well, it sucks.", "start": 2396.173, "speaker": "SPEAKER_05"}, {"end": 2399.054, "text": "That's what's wrong with it.", "start": 2397.634, "speaker": "SPEAKER_05"}, {"end": 2400.915, "text": "So most people forget a password.", "start": 2399.054, "speaker": "SPEAKER_05"}, {"end": 2403.255, "text": "I don't know how that's possible, but they do.", "start": 2400.915, "speaker": "SPEAKER_05"}, {"end": 2409.877, "text": "5% to 10% of sessions require people going through that heinous request password loop because they forget.", "start": 2403.255, "speaker": "SPEAKER_05"}, {"end": 2416.64, "text": "The number one request to internet help desks, the number one thing that internal companies deal with is password issues.", "start": 2409.877, "speaker": "SPEAKER_05"}, {"end": 2419.06, "text": "So this thing actually sucks.", "start": 2416.64, "speaker": "SPEAKER_05"}, {"end": 2423.722, "text": "It has tons of problems, lots of data out there that shows logging in is a pain in the butt.", "start": 2419.06, "speaker": "SPEAKER_05"}, {"end": 2433.149, "text": "So one of the reasons why logging in is a pain in the butt, because you can't actually see what the heck you're putting in there.", "start": 2425.363, "speaker": "SPEAKER_05"}, {"end": 2436.351, "text": "It's just dot, dot, dot, dot, dot, dot, dot.", "start": 2433.149, "speaker": "SPEAKER_05"}, {"end": 2442.796, "text": "In the interest of security, somebody could be looking over your shoulder.", "start": 2436.351, "speaker": "SPEAKER_05"}, {"end": 2443.716, "text": "Especially on mobile.", "start": 2442.796, "speaker": "SPEAKER_05"}, {"end": 2449.661, "text": "If there's a guy in a trench coat and a mustache behind you, just go like this.", "start": 2443.716, "speaker": "SPEAKER_05"}, {"end": 2450.481, "text": "There's your security.", "start": 2449.661, "speaker": "SPEAKER_05"}, {"end": 2462.774, "text": "Not being able to see what you're typing is actually a huge issue when you consider what people make you do to passwords these days.", "start": 2454.91, "speaker": "SPEAKER_05"}, {"end": 2470.678, "text": "So brief aside, call it a rant if you must, I am a registered member of this comic book subscription service, which I turn to to relax.", "start": 2462.774, "speaker": "SPEAKER_05"}, {"end": 2473.644, "text": "Comic books are fun.", "start": 2472.183, "speaker": "SPEAKER_05"}, {"end": 2475.846, "text": "They're supposed to be fun.", "start": 2473.644, "speaker": "SPEAKER_05"}, {"end": 2481.99, "text": "And so I get this message one day, as an added security measure, we have strengthened password requirements.", "start": 2475.846, "speaker": "SPEAKER_05"}, {"end": 2488.194, "text": "If you have not done so, you must create a new password that meets our updated requirements.", "start": 2481.99, "speaker": "SPEAKER_05"}, {"end": 2491.777, "text": "I just want to read The Hulk, dude, right?", "start": 2488.194, "speaker": "SPEAKER_05"}, {"end": 2494.619, "text": "And like, seriously, what if somebody reads The Hulk instead of me?", "start": 2491.777, "speaker": "SPEAKER_05"}, {"end": 2498.782, "text": "Does it really matter?", "start": 2494.619, "speaker": "SPEAKER_05"}, {"end": 2499.302, "text": "But whatever.", "start": 2498.782, "speaker": "SPEAKER_05"}, {"end": 2502.944, "text": "I get locked out.", "start": 2501.523, "speaker": "SPEAKER_05"}, {"end": 2503.884, "text": "I can't read anything.", "start": 2502.944, "speaker": "SPEAKER_05"}, {"end": 2506.986, "text": "First, it took me a while to figure out what's going on, because the whole app broke.", "start": 2503.884, "speaker": "SPEAKER_05"}, {"end": 2511.128, "text": "And then finally, I somehow made my way to try and log in, and I saw this message.", "start": 2506.986, "speaker": "SPEAKER_05"}, {"end": 2513.229, "text": "All right, I guess I got to create a new password.", "start": 2511.128, "speaker": "SPEAKER_05"}, {"end": 2517.172, "text": "And then come the password rules.", "start": 2513.229, "speaker": "SPEAKER_05"}, {"end": 2525.076, "text": "Eight characters, one capital letter, one lowercase letter, one number, one special character, bang, hash, dollar, percent, what have you.", "start": 2517.172, "speaker": "SPEAKER_05"}, {"end": 2529.826, "text": "How much fun is that?", "start": 2528.166, "speaker": "SPEAKER_05"}, {"end": 2530.807, "text": "Not fun at all.", "start": 2529.826, "speaker": "SPEAKER_05"}, {"end": 2535.868, "text": "And so we get forced to make these things that no human being could possibly ever remember.", "start": 2530.807, "speaker": "SPEAKER_05"}, {"end": 2541.309, "text": "And then when we get told to log in, we can't even see the damn thing.", "start": 2535.868, "speaker": "SPEAKER_05"}, {"end": 2543.61, "text": "So passwords get really complicated.", "start": 2541.309, "speaker": "SPEAKER_05"}, {"end": 2549.791, "text": "And so it's really nice when apps actually let you see what you're typing in, so you don't make mistakes, so you can get in.", "start": 2543.61, "speaker": "SPEAKER_05"}, {"end": 2553.012, "text": "And as we saw earlier, those mistakes can be pretty costly to support.", "start": 2549.791, "speaker": "SPEAKER_05"}, {"end": 2561.623, "text": "So Amazon, in their sign-in screen, introduced this feature that when you type in your password, you can tap that.", "start": 2554.035, "speaker": "SPEAKER_05"}, {"end": 2562.925, "text": "Oops, let me come back.", "start": 2561.623, "speaker": "SPEAKER_05"}, {"end": 2565.227, "text": "I guess I didn't spend all night on this one.", "start": 2562.925, "speaker": "SPEAKER_05"}, {"end": 2570.413, "text": "When you tap this show password thing, you can actually see what your password looks like, and you can untap to look at it.", "start": 2565.227, "speaker": "SPEAKER_05"}, {"end": 2576.087, "text": "And this isn't just a nice city on behalf of Amazon, it actually adds up to real revenue.", "start": 2571.585, "speaker": "SPEAKER_05"}, {"end": 2587.37, "text": "It turns out that inside of the shopping cart flow, if somebody tries to log in and they enter a password recovery loop, three-fourths of the time, they will not complete the purchase.", "start": 2576.087, "speaker": "SPEAKER_05"}, {"end": 2592.352, "text": "So in many ways, any time you're basically sending somebody off to recover a password, you're bleeding money.", "start": 2587.37, "speaker": "SPEAKER_05"}, {"end": 2599.054, "text": "And this show password pattern, Etsy launched it for a while on their e-commerce site.", "start": 2593.971, "speaker": "SPEAKER_05"}, {"end": 2604.777, "text": "They showed that this actually decreases failed logins, which therefore helps you actually grow your conversion.", "start": 2599.054, "speaker": "SPEAKER_05"}, {"end": 2606.458, "text": "So there's actual money behind this.", "start": 2604.777, "speaker": "SPEAKER_05"}, {"end": 2611.001, "text": "It's not just being nice to human beings to allow them to see your password.", "start": 2606.458, "speaker": "SPEAKER_05"}, {"end": 2616.064, "text": "It actually reduces failed logins, which helps grow the amount of people that can actually buy stuff.", "start": 2611.001, "speaker": "SPEAKER_05"}, {"end": 2618.289, "text": "And why does this work?", "start": 2617.168, "speaker": "SPEAKER_05"}, {"end": 2622.832, "text": "Well, looking at some studies of how people interact with things, the pattern works like this.", "start": 2618.289, "speaker": "SPEAKER_05"}, {"end": 2628.355, "text": "You start typing some dots, you hit show, then you can see the password, and then you can type the rest of it in.", "start": 2622.832, "speaker": "SPEAKER_05"}, {"end": 2630.5018131868133, "text": "You know, some of the testing that's... ", "start": 2628.355, "speaker": "SPEAKER_05"}, {"end": 2633.239, "text": "By the way, all these things are linked in sources.", "start": 2630.5018131868133, "speaker": "SPEAKER_05"}, {"end": 2635.94, "text": "I'll give you the slides later so you can take a look at them all.", "start": 2633.239, "speaker": "SPEAKER_05"}, {"end": 2645.807, "text": "But in testing, people, once they start typing the first character, they'll go and unmask the password, and they kind of understand that interaction to go and see stuff.", "start": 2635.94, "speaker": "SPEAKER_05"}, {"end": 2647.968, "text": "Which is cool.", "start": 2647.148, "speaker": "SPEAKER_05"}, {"end": 2651.33, "text": "So this is useful to help people log in, yay.", "start": 2647.968, "speaker": "SPEAKER_05"}, {"end": 2654.171, "text": "But that method kind of requires some effort on the user's part.", "start": 2651.33, "speaker": "SPEAKER_05"}, {"end": 2659.914, "text": "They actually have to notice that show button, they have to tap on the show icon to get the benefit of this feature.", "start": 2654.171, "speaker": "SPEAKER_05"}, {"end": 2668.338, "text": "So in the next iteration that Amazon went through, they changed sort of the dynamics and now the password's revealed by default.", "start": 2659.914, "speaker": "SPEAKER_05"}, {"end": 2671.9, "text": "So as you start typing, you see your password and you have the option to hide it.", "start": 2668.338, "speaker": "SPEAKER_05"}, {"end": 2673.873, "text": "and remove it.", "start": 2673.072, "speaker": "SPEAKER_05"}, {"end": 2678.978, "text": "You can see what you're typing, yay, and then you can say, please don't show that.", "start": 2673.873, "speaker": "SPEAKER_05"}, {"end": 2686.306, "text": "And again, this makes sense too, because if you simply show people's passwords by default, you muck with their heads.", "start": 2678.978, "speaker": "SPEAKER_05"}, {"end": 2694.155, "text": "We have unfortunately trained so many people to think those little dots are secure, which that's all I'm going to say.", "start": 2686.306, "speaker": "SPEAKER_05"}, {"end": 2696.544, "text": "But they think they are.", "start": 2695.543, "speaker": "SPEAKER_05"}, {"end": 2700.85, "text": "So you muck with people's expectations when you just show them their password by default.", "start": 2696.544, "speaker": "SPEAKER_05"}, {"end": 2709.16, "text": "So if you just type your password like this, again, looking at data, most people do not expect to see their password as clear text.", "start": 2700.85, "speaker": "SPEAKER_05"}, {"end": 2712.204, "text": "And many of them become suspicious.", "start": 2709.16, "speaker": "SPEAKER_05"}, {"end": 2713.966, "text": "Hmm.", "start": 2712.204, "speaker": "SPEAKER_05"}, {"end": 2718.646, "text": "So how do you fix this?", "start": 2717.385, "speaker": "SPEAKER_05"}, {"end": 2728.252, "text": "Well, you do kind of what Amazon did, adding this very simple little control, this show password dialogue here, lets people know what's happening and helps mitigate this issue.", "start": 2718.646, "speaker": "SPEAKER_05"}, {"end": 2734.616, "text": "This is a pretty small change relative to the way changes go, but oftentimes it's these small changes that really add up.", "start": 2728.252, "speaker": "SPEAKER_05"}, {"end": 2744.803, "text": "I think Craig made that point at the end of the day yesterday with lots of swear words, but hopefully it got through, which is little things can come together to make a big impact.", "start": 2734.616, "speaker": "SPEAKER_05"}, {"end": 2755.092, "text": "And in this case, instead of thinking this is an error now, 100% of people notice that checkbox, and now they think, oh, this site is watching out for me.", "start": 2746.298, "speaker": "SPEAKER_05"}, {"end": 2757.135, "text": "This is an improvement in user experience.", "start": 2755.092, "speaker": "SPEAKER_05"}, {"end": 2759.278, "text": "It's trying to help me log in accurately.", "start": 2757.135, "speaker": "SPEAKER_05"}, {"end": 2760.28, "text": "Thank you, website.", "start": 2759.278, "speaker": "SPEAKER_05"}, {"end": 2760.841, "text": "I enjoy you.", "start": 2760.28, "speaker": "SPEAKER_05"}, {"end": 2767.299, "text": "Taking this a step further, and note the iteration here.", "start": 2764.078, "speaker": "SPEAKER_05"}, {"end": 2769.34, "text": "This is still the Amazon login screen.", "start": 2767.299, "speaker": "SPEAKER_05"}, {"end": 2772.041, "text": "This is like version four of just login.", "start": 2769.34, "speaker": "SPEAKER_05"}, {"end": 2779.363, "text": "They now have a visible version of the password below this hidden version of the password.", "start": 2772.041, "speaker": "SPEAKER_05"}, {"end": 2781.164, "text": "Again, playing into a lot of the stuff that we saw.", "start": 2779.363, "speaker": "SPEAKER_05"}, {"end": 2789.607, "text": "So you can see that your password is still quote unquote secure with the dots, but you can actually see what you're typing that's default on and you can hide it if you so choose.", "start": 2781.164, "speaker": "SPEAKER_05"}, {"end": 2797.231, "text": "So several iterations later, this is the UI that's out there right now.", "start": 2791.447, "speaker": "SPEAKER_05"}, {"end": 2800.053, "text": "And I think this is an important thing.", "start": 2797.231, "speaker": "SPEAKER_05"}, {"end": 2802.315, "text": "I think sign-in screens matter, right?", "start": 2800.053, "speaker": "SPEAKER_05"}, {"end": 2806.218, "text": "Getting people logged in, authenticated, creates business, you name it.", "start": 2802.315, "speaker": "SPEAKER_05"}, {"end": 2809, "text": "And so iterating on it makes sense.", "start": 2806.218, "speaker": "SPEAKER_05"}, {"end": 2811.082, "text": "First, you couldn't view your password.", "start": 2809, "speaker": "SPEAKER_05"}, {"end": 2812.883, "text": "Then you could tap to reveal it.", "start": 2811.082, "speaker": "SPEAKER_05"}, {"end": 2815.905, "text": "Then it was revealed by default, but you could tap to hide it.", "start": 2812.883, "speaker": "SPEAKER_05"}, {"end": 2819.728, "text": "Now it's revealed below as you type and defaulted on, but you can remove it.", "start": 2815.905, "speaker": "SPEAKER_05"}, {"end": 2827.635, "text": "So is this problem clearly important enough for someone like Amazon to iterate on a lot?", "start": 2822.092, "speaker": "SPEAKER_05"}, {"end": 2835.56, "text": "But are they really taking this in a future thinking way?", "start": 2827.635, "speaker": "SPEAKER_05"}, {"end": 2841.144, "text": "Is this more of the same stuff we saw with the 30 years of Illustrator?", "start": 2835.56, "speaker": "SPEAKER_05"}, {"end": 2846.327, "text": "Are we just moving pixels around and getting maybe like a 1% boost here, 2% boost there kind of thing?", "start": 2841.144, "speaker": "SPEAKER_05"}, {"end": 2856.494, "text": "Are we optimizing for a local maximum, or are we really moving towards this future that I've been talking about that mobile enables?", "start": 2849.412, "speaker": "SPEAKER_05"}, {"end": 2857.555, "text": "And what's the difference, right?", "start": 2856.494, "speaker": "SPEAKER_05"}, {"end": 2861.736, "text": "What's the difference between iterating and actually taking advantage of the mobile opportunity?", "start": 2857.555, "speaker": "SPEAKER_05"}, {"end": 2868.478, "text": "Well, following all these design updates, Amazon launched the ability to sign in with just a fingerprint on iOS.", "start": 2861.736, "speaker": "SPEAKER_05"}, {"end": 2875.261, "text": "They use Touch ID, and you don't type in a password or a username or anything.", "start": 2868.478, "speaker": "SPEAKER_05"}, {"end": 2882.237, "text": "You just kind of put your thumb on a little button And you're logged in.", "start": 2875.261, "speaker": "SPEAKER_05"}, {"end": 2887.662, "text": "And I think this is kind of a nice small example of what I'm talking about.", "start": 2882.237, "speaker": "SPEAKER_05"}, {"end": 2889.523, "text": "What's the vision?", "start": 2887.662, "speaker": "SPEAKER_05"}, {"end": 2895.829, "text": "Is the vision, let's gradually keep making sign-in better bit by bit by bit over a 10-year period?", "start": 2889.523, "speaker": "SPEAKER_05"}, {"end": 2897.37, "text": "Or is the vision, no sign-in?", "start": 2895.829, "speaker": "SPEAKER_05"}, {"end": 2903.721, "text": "Mobile devices have a ton of amazing capabilities.", "start": 2899.7, "speaker": "SPEAKER_05"}, {"end": 2908.282, "text": "They've got biometric readers for authentication.", "start": 2903.721, "speaker": "SPEAKER_05"}, {"end": 2919.245, "text": "They have built-in sensors, microphones, cameras, geolocation, digital compasses, accelerometers, you name it.", "start": 2908.282, "speaker": "SPEAKER_05"}, {"end": 2925.007, "text": "There's a ton of stuff in here in terms of capabilities that we can start to use to really change the game.", "start": 2919.245, "speaker": "SPEAKER_05"}, {"end": 2926.407, "text": "And so iteration is awesome.", "start": 2925.007, "speaker": "SPEAKER_05"}, {"end": 2934.14, "text": "I definitively believe this has made the Amazon experience better, probably in measurable ways.", "start": 2927.395, "speaker": "SPEAKER_05"}, {"end": 2941.605, "text": "But we also need a vision, one that is somewhere far out, something that changes the game, a vision of what the future can be.", "start": 2934.14, "speaker": "SPEAKER_05"}, {"end": 2946.468, "text": "And ideally, it's kind of rooted in the capabilities of what these things can do.", "start": 2941.605, "speaker": "SPEAKER_05"}, {"end": 2948.77, "text": "For example, oh, we've got fingerprint sensors.", "start": 2946.468, "speaker": "SPEAKER_05"}, {"end": 2950.511, "text": "OK, let's just off with a single tap.", "start": 2948.77, "speaker": "SPEAKER_05"}, {"end": 2958.215, "text": "So as we go about optimizing for today, and there's a lot of things we can do to optimize for today, how do we think about tomorrow?", "start": 2951.391, "speaker": "SPEAKER_05"}, {"end": 2959.496, "text": "What is this North Star?", "start": 2958.215, "speaker": "SPEAKER_05"}, {"end": 2962.818, "text": "Where do we want to get to?", "start": 2959.496, "speaker": "SPEAKER_05"}, {"end": 2969.262, "text": "Taking this a little bit further, so Uber, which is a service you probably all know, some of you may like it, some of you may not like it.", "start": 2962.818, "speaker": "SPEAKER_05"}, {"end": 2977.676, "text": "But they have an online registration form to create an account, make a profile, provide payment information.", "start": 2970.152, "speaker": "SPEAKER_05"}, {"end": 2978.717, "text": "And it's pretty typical.", "start": 2977.676, "speaker": "SPEAKER_05"}, {"end": 2980.378, "text": "You make an account and a password.", "start": 2978.717, "speaker": "SPEAKER_05"}, {"end": 2986.441, "text": "You enter a profile, phone information, enter your payment info, submit to their terms of service, as you do.", "start": 2980.378, "speaker": "SPEAKER_05"}, {"end": 2989.913, "text": "on the internet, and it's pretty well designed.", "start": 2987.912, "speaker": "SPEAKER_05"}, {"end": 2994.256, "text": "They've taken a bunch of best practices into account here.", "start": 2989.913, "speaker": "SPEAKER_05"}, {"end": 3004.603, "text": "So they have this typical sign-up register form, but they also have a complete registration and payment flow using just Touch ID, as we saw on Amazon a second ago.", "start": 2994.256, "speaker": "SPEAKER_05"}, {"end": 3013.47, "text": "And here, what you do is just one tap on that button, and you immediately create an account and pay for your ride, and that's it.", "start": 3004.603, "speaker": "SPEAKER_05"}, {"end": 3013.77, "text": "You're done.", "start": 3013.47, "speaker": "SPEAKER_05"}, {"end": 3015.703, "text": "There's no sign-in form.", "start": 3014.643, "speaker": "SPEAKER_05"}, {"end": 3016.764, "text": "There's no check-in form.", "start": 3015.703, "speaker": "SPEAKER_05"}, {"end": 3017.964, "text": "There's no payment info.", "start": 3016.764, "speaker": "SPEAKER_05"}, {"end": 3019.385, "text": "There's nothing.", "start": 3017.964, "speaker": "SPEAKER_05"}, {"end": 3021.026, "text": "You just go like this.", "start": 3019.385, "speaker": "SPEAKER_05"}, {"end": 3026.288, "text": "You're now a customer, and you've paid for the cab that's coming to get you.", "start": 3021.026, "speaker": "SPEAKER_05"}, {"end": 3030.65, "text": "And when we look at this, what do you think the future is?", "start": 3026.288, "speaker": "SPEAKER_05"}, {"end": 3038.633, "text": "And this is why I sort of push back against questions like, our desktop conversion is better than our mobile conversion.", "start": 3030.65, "speaker": "SPEAKER_05"}, {"end": 3042.415, "text": "It's because you're taking desktop forms and shoving them onto little mobile screens.", "start": 3038.633, "speaker": "SPEAKER_05"}, {"end": 3050.93, "text": "Speaking of payments, which are this key part of conversion, let's take a step back and look at that, exactly what I was just talking about.", "start": 3046.485, "speaker": "SPEAKER_05"}, {"end": 3054.113, "text": "This is the process of collecting payment info.", "start": 3050.93, "speaker": "SPEAKER_05"}, {"end": 3065.726, "text": "10 years later, 10 years after this onset of all these devices and all the learnings and impact we've had, our framing of payments is still rooted in desktop checkout forms, this kind of stuff.", "start": 3054.113, "speaker": "SPEAKER_05"}, {"end": 3074.953, "text": "Now granted, simply taking a desktop checkout form and making it work for a variety of different devices actually pays dividends.", "start": 3066.985, "speaker": "SPEAKER_05"}, {"end": 3087.427, "text": "So when Walmart in Canada created a responsive design for their e-commerce site where it would now work on small, medium, and medium-large and medium-small type devices, they saw a big impact.", "start": 3074.953, "speaker": "SPEAKER_05"}, {"end": 3091.43, "text": "They got increases in conversion, especially on mobile.", "start": 3087.907, "speaker": "SPEAKER_05"}, {"end": 3094.513, "text": "And I think Ani talked about this yesterday for Alibaba.", "start": 3091.43, "speaker": "SPEAKER_05"}, {"end": 3097.996, "text": "When they move to a responsive design, good things happen.", "start": 3094.513, "speaker": "SPEAKER_05"}, {"end": 3105.142, "text": "So just making something accessible on a variety of devices can help use, because you can now use it on those devices.", "start": 3097.996, "speaker": "SPEAKER_05"}, {"end": 3107.945, "text": "That's a good thing.", "start": 3105.142, "speaker": "SPEAKER_05"}, {"end": 3115.722, "text": "However, the entirety of what worked on desktop isn't necessarily the best fit for mobile.", "start": 3107.945, "speaker": "SPEAKER_05"}, {"end": 3125.025, "text": "So when we look at Dell's checkout experience, you can see there's the shipping section, the payment section, the verify section, the order submitted section, all this stuff.", "start": 3115.722, "speaker": "SPEAKER_05"}, {"end": 3137.83, "text": "Taking all of these things and putting them on mobile, all of it on mobile, and I really mean all of it, is not necessarily the best experience.", "start": 3125.025, "speaker": "SPEAKER_05"}, {"end": 3141.231, "text": "I mean, I love the fact that they're still collecting fax numbers here.", "start": 3137.83, "speaker": "SPEAKER_05"}, {"end": 3144.153, "text": "There it is.", "start": 3143.372, "speaker": "SPEAKER_05"}, {"end": 3146.995, "text": "Daytime fax number with an extension.", "start": 3144.153, "speaker": "SPEAKER_05"}, {"end": 3148.716, "text": "Of course, that makes sense.", "start": 3146.995, "speaker": "SPEAKER_05"}, {"end": 3155.721, "text": "And once I've filled in all this information, I absolutely want to not agree to your terms of service.", "start": 3148.716, "speaker": "SPEAKER_05"}, {"end": 3160.244, "text": "And on and on and on, and so it goes.", "start": 3155.721, "speaker": "SPEAKER_05"}, {"end": 3162.065, "text": "Oh, my conversion on mobile sucks.", "start": 3160.244, "speaker": "SPEAKER_05"}, {"end": 3164.547, "text": "Duh.", "start": 3162.065, "speaker": "SPEAKER_05"}, {"end": 3167.549, "text": "I'm surprised anybody ever filled it out.", "start": 3164.547, "speaker": "SPEAKER_05"}, {"end": 3171.592, "text": "You should be counting your blessings that a human being made it through this flow.", "start": 3167.549, "speaker": "SPEAKER_05"}, {"end": 3179.454, "text": "God bless that human.", "start": 3175.673, "speaker": "SPEAKER_05"}, {"end": 3182.155, "text": "So there's a few, perhaps, obvious reasons this is suboptimal.", "start": 3179.454, "speaker": "SPEAKER_05"}, {"end": 3183.255, "text": "For one, perception.", "start": 3182.155, "speaker": "SPEAKER_05"}, {"end": 3192.077, "text": "If something seems onerous, I think the conversation yesterday around cognitive load illustrates this.", "start": 3183.255, "speaker": "SPEAKER_05"}, {"end": 3195.638, "text": "So this looks freaking hard.", "start": 3192.077, "speaker": "SPEAKER_05"}, {"end": 3197.219, "text": "Ugh, I don't want to do that.", "start": 3195.638, "speaker": "SPEAKER_05"}, {"end": 3199.359, "text": "That's terrible.", "start": 3197.219, "speaker": "SPEAKER_05"}, {"end": 3204.571, "text": "So processes that seem onerous to people they'll avoid, because these are onerous.", "start": 3199.359, "speaker": "SPEAKER_05"}, {"end": 3210.673, "text": "When people see this, they're going to choose the easiest path forward, which is abandonment, going away.", "start": 3204.571, "speaker": "SPEAKER_05"}, {"end": 3217.916, "text": "Whereas something like this, which actually captures a lot of the same information, feels, oh, I can do that.", "start": 3210.673, "speaker": "SPEAKER_05"}, {"end": 3219.356, "text": "I can make my way through this.", "start": 3217.916, "speaker": "SPEAKER_05"}, {"end": 3228.387, "text": "So that's kind of one thing, taking the time to create this more focused, obvious form can go a long way to addressing the perception issue.", "start": 3219.356, "speaker": "SPEAKER_05"}, {"end": 3229.768, "text": "But there's other issues.", "start": 3228.387, "speaker": "SPEAKER_05"}, {"end": 3234.57, "text": "In addition to our instinctive feelings of complexity, you can actually measure the benefits.", "start": 3229.768, "speaker": "SPEAKER_05"}, {"end": 3238.272, "text": "So those two examples that I showed earlier, I've been showing them here and there.", "start": 3234.57, "speaker": "SPEAKER_05"}, {"end": 3240.433, "text": "And somebody went and tested them against each other.", "start": 3238.272, "speaker": "SPEAKER_05"}, {"end": 3247.437, "text": "So they took that kind of multi-dropdown form for flight travel to the simplified version.", "start": 3240.433, "speaker": "SPEAKER_05"}, {"end": 3252.219, "text": "And they found that completion times Huge improvement.", "start": 3247.437, "speaker": "SPEAKER_05"}, {"end": 3257.866, "text": "People were able to make it through this thing much faster and easier than they were in the last one, which pays off.", "start": 3252.219, "speaker": "SPEAKER_05"}, {"end": 3266.497, "text": "So in addition to perception, there's sort of these measurable benefits of improved interactions, completion times yay, and of course, the one we all care about, conversion rate impact.", "start": 3257.866, "speaker": "SPEAKER_05"}, {"end": 3272.726, "text": "And for a while, I've been showing this Boingo redesign form because I like it for a couple of reasons.", "start": 3267.263, "speaker": "SPEAKER_05"}, {"end": 3277.108, "text": "A, they sent me a redesign which had a real nice impact on their overall conversion.", "start": 3272.726, "speaker": "SPEAKER_05"}, {"end": 3282.27, "text": "When they dropped 11 input fields, they got a 34% boost.", "start": 3277.108, "speaker": "SPEAKER_05"}, {"end": 3287.273, "text": "And again, same sort of decrease in effort that we saw before.", "start": 3282.27, "speaker": "SPEAKER_05"}, {"end": 3293.696, "text": "But I like it more because I actually had a design that takes it down to four inputs that I thought would be a bit more effective.", "start": 3287.273, "speaker": "SPEAKER_05"}, {"end": 3300.969, "text": "And now, what's really awesome is a couple of sites have gone and implemented things that look a lot like this.", "start": 3294.701, "speaker": "SPEAKER_05"}, {"end": 3305.133, "text": "And I actually have real-world examples from very large e-commerce retailers.", "start": 3300.969, "speaker": "SPEAKER_05"}, {"end": 3307.596, "text": "This is Staples in the United States.", "start": 3305.133, "speaker": "SPEAKER_05"}, {"end": 3314.925, "text": "And they basically took a similar 22-input screen and got it down to just five.", "start": 3307.596, "speaker": "SPEAKER_05"}, {"end": 3319.933, "text": "from 22 things straight down to five.", "start": 3316.972, "speaker": "SPEAKER_05"}, {"end": 3323.153, "text": "And so before when I used to show this, I'm like, I can do it in four.", "start": 3319.933, "speaker": "SPEAKER_05"}, {"end": 3325.093, "text": "And people were like, whatever.", "start": 3323.153, "speaker": "SPEAKER_05"}, {"end": 3327.514, "text": "Now I can say, look, they did it in five.", "start": 3325.093, "speaker": "SPEAKER_05"}, {"end": 3328.194, "text": "It's real.", "start": 3327.514, "speaker": "SPEAKER_05"}, {"end": 3332.395, "text": "This is not a designer dream kind of thing.", "start": 3328.194, "speaker": "SPEAKER_05"}, {"end": 3335.435, "text": "And so I think it's worth looking at this in depth.", "start": 3332.395, "speaker": "SPEAKER_05"}, {"end": 3345.357, "text": "Again, as part of similar to what we did with login, what are the optimizations you can do today to work for these kinds of devices in these situations?", "start": 3335.435, "speaker": "SPEAKER_05"}, {"end": 3348.749, "text": "So let's do that.", "start": 3347.208, "speaker": "SPEAKER_05"}, {"end": 3352.031, "text": "Step one here is in this checkout.", "start": 3348.749, "speaker": "SPEAKER_05"}, {"end": 3353.692, "text": "And by the way, this is guest checkout.", "start": 3352.031, "speaker": "SPEAKER_05"}, {"end": 3357.214, "text": "If you don't have an explicit guest checkout, you're probably missing out.", "start": 3353.692, "speaker": "SPEAKER_05"}, {"end": 3362.997, "text": "A guest checkout has been shown to boost the amount of people that can use things by somewhere 30%, 40%.", "start": 3357.214, "speaker": "SPEAKER_05"}, {"end": 3365.919, "text": "Because in most cases, people don't want to get married to a service.", "start": 3362.997, "speaker": "SPEAKER_05"}, {"end": 3370.182, "text": "They just want to get in and out.", "start": 3365.919, "speaker": "SPEAKER_05"}, {"end": 3373.183, "text": "So step one is capturing the customer's name.", "start": 3370.182, "speaker": "SPEAKER_05"}, {"end": 3375.605, "text": "And you'll notice that there is just a single input field.", "start": 3373.183, "speaker": "SPEAKER_05"}, {"end": 3378.033, "text": "for the customer's name.", "start": 3376.992, "speaker": "SPEAKER_05"}, {"end": 3381.355, "text": "And there's a couple reasons behind this, so let me dig into this.", "start": 3378.033, "speaker": "SPEAKER_05"}, {"end": 3385.899, "text": "And we'll walk through each of these five fields and talk about what's happening.", "start": 3381.355, "speaker": "SPEAKER_05"}, {"end": 3388.861, "text": "But basically, the principle here is avoiding splitting input fields.", "start": 3385.899, "speaker": "SPEAKER_05"}, {"end": 3392.724, "text": "And once people are in typing mode, just let them stay in input mode.", "start": 3388.861, "speaker": "SPEAKER_05"}, {"end": 3396.006, "text": "Don't make them jump between fields in input mode too much.", "start": 3392.724, "speaker": "SPEAKER_05"}, {"end": 3399.929, "text": "There's a couple reasons why we should avoid splitting input fields.", "start": 3396.006, "speaker": "SPEAKER_05"}, {"end": 3403.312, "text": "And Baymert Institute has a real nice set of reasons around this.", "start": 3399.929, "speaker": "SPEAKER_05"}, {"end": 3406.438, "text": "which I'll kind of walk through here.", "start": 3404.617, "speaker": "SPEAKER_05"}, {"end": 3408.719, "text": "So people have a hard time moving between fields.", "start": 3406.438, "speaker": "SPEAKER_05"}, {"end": 3411.081, "text": "Sometimes those divisions don't make sense.", "start": 3408.719, "speaker": "SPEAKER_05"}, {"end": 3414.122, "text": "And sometimes it's unclear what is or isn't required.", "start": 3411.081, "speaker": "SPEAKER_05"}, {"end": 3416.824, "text": "So we'll start with the bottom one here.", "start": 3414.122, "speaker": "SPEAKER_05"}, {"end": 3420.406, "text": "When we look at what is or isn't required, of course, you know the asterisks, right?", "start": 3416.824, "speaker": "SPEAKER_05"}, {"end": 3423.367, "text": "Everybody knows the asterisks.", "start": 3420.406, "speaker": "SPEAKER_05"}, {"end": 3424.848, "text": "We can talk about that later.", "start": 3423.367, "speaker": "SPEAKER_05"}, {"end": 3429.15, "text": "But what's required in zip code?", "start": 3424.848, "speaker": "SPEAKER_05"}, {"end": 3431.832, "text": "Both of these or one of these?", "start": 3429.15, "speaker": "SPEAKER_05"}, {"end": 3436.739, "text": "Makes you think, doesn't it?", "start": 3435.079, "speaker": "SPEAKER_05"}, {"end": 3439.44, "text": "You don't know, do you?", "start": 3436.739, "speaker": "SPEAKER_05"}, {"end": 3440.96, "text": "You can guess, can't you?", "start": 3439.44, "speaker": "SPEAKER_05"}, {"end": 3442.861, "text": "But you don't know.", "start": 3440.96, "speaker": "SPEAKER_05"}, {"end": 3445.741, "text": "And this is an example of that issue that we talked about here.", "start": 3442.861, "speaker": "SPEAKER_05"}, {"end": 3447.002, "text": "It's unclear what's required.", "start": 3445.741, "speaker": "SPEAKER_05"}, {"end": 3457.544, "text": "So when you split input fields and you have this situation where certain things are required, certain things are optional, that split field doesn't really help people know what they have to do.", "start": 3447.002, "speaker": "SPEAKER_05"}, {"end": 3459.265, "text": "And if it is required, it can lead to errors.", "start": 3457.544, "speaker": "SPEAKER_05"}, {"end": 3461.265, "text": "If it's not required, you get the idea, right?", "start": 3459.265, "speaker": "SPEAKER_05"}, {"end": 3465.144, "text": "So this is one of the problems with splitting input fields.", "start": 3462.302, "speaker": "SPEAKER_05"}, {"end": 3469.708, "text": "The other problem with splitting input fields, people have a hard time moving between fields.", "start": 3465.144, "speaker": "SPEAKER_05"}, {"end": 3471.37, "text": "So this person switches.", "start": 3469.708, "speaker": "SPEAKER_05"}, {"end": 3473.071, "text": "Let's look at this from the beginning here.", "start": 3471.37, "speaker": "SPEAKER_05"}, {"end": 3476.434, "text": "The person switches to the numeric keypad.", "start": 3473.071, "speaker": "SPEAKER_05"}, {"end": 3476.974, "text": "OK, great.", "start": 3476.434, "speaker": "SPEAKER_05"}, {"end": 3478.896, "text": "Now they type in three digits of their phone number.", "start": 3476.974, "speaker": "SPEAKER_05"}, {"end": 3480.998, "text": "And now it auto tabs, so the keyboard goes down.", "start": 3478.896, "speaker": "SPEAKER_05"}, {"end": 3484.421, "text": "So now I have to tap the field again, and they have to switch it over to the number field again.", "start": 3480.998, "speaker": "SPEAKER_05"}, {"end": 3485.662, "text": "Now they're back in the number field.", "start": 3484.421, "speaker": "SPEAKER_05"}, {"end": 3496.953, "text": "They can type the next three digits, and then, of course, it auto-taps, so it goes down, and they have to go back to open the keyboard, switch over to the number field, tap in the number, and then, finally, they get through that field.", "start": 3486.042, "speaker": "SPEAKER_05"}, {"end": 3498.355, "text": "What a pain in the ass, right?", "start": 3496.953, "speaker": "SPEAKER_05"}, {"end": 3510.746, "text": "And so taking your desktop forms that work on mobile and putting them on these devices makes no sense because it is a phone.", "start": 3500.382, "speaker": "SPEAKER_05"}, {"end": 3512.587, "text": "We can just capture phone numbers like this.", "start": 3510.746, "speaker": "SPEAKER_05"}, {"end": 3517.429, "text": "And if you really need secondary info, there's ways to capture it, what have you.", "start": 3512.587, "speaker": "SPEAKER_05"}, {"end": 3519.95, "text": "But the same principle applies to first name, last name.", "start": 3517.429, "speaker": "SPEAKER_05"}, {"end": 3524.612, "text": "If you've ever observed a usability test, you've probably seen somebody do this.", "start": 3519.95, "speaker": "SPEAKER_05"}, {"end": 3529.068, "text": "If you've ever watched somebody fill in a form, I've seen this many, many times.", "start": 3524.612, "speaker": "SPEAKER_05"}, {"end": 3533.451, "text": "People will see a name field, and they'll just enter their first name and their last name, because that's their name.", "start": 3529.068, "speaker": "SPEAKER_05"}, {"end": 3535.032, "text": "You just start typing their name.", "start": 3533.451, "speaker": "SPEAKER_05"}, {"end": 3536.233, "text": "You've probably done this, right?", "start": 3535.032, "speaker": "SPEAKER_05"}, {"end": 3537.453, "text": "You go in there, you say, oh, crap.", "start": 3536.233, "speaker": "SPEAKER_05"}, {"end": 3539.435, "text": "Oh, I've got to separate name.", "start": 3537.453, "speaker": "SPEAKER_05"}, {"end": 3543.497, "text": "That's because your name to you is a single coherent entity.", "start": 3539.435, "speaker": "SPEAKER_05"}, {"end": 3548.941, "text": "And this is the example of that earlier comment from Behmer.", "start": 3543.497, "speaker": "SPEAKER_05"}, {"end": 3550.602, "text": "People find division illogical.", "start": 3548.941, "speaker": "SPEAKER_05"}, {"end": 3558.775, "text": "meaning this is my name, I will enter my name, I don't think of my name in necessarily these parts until you tell me to.", "start": 3551.648, "speaker": "SPEAKER_05"}, {"end": 3562.078, "text": "And so a single name is also more representative of global differences.", "start": 3558.775, "speaker": "SPEAKER_05"}, {"end": 3566.962, "text": "There's people who have a single name, there's people who have eight names.", "start": 3562.078, "speaker": "SPEAKER_05"}, {"end": 3567.503, "text": "True story.", "start": 3566.962, "speaker": "SPEAKER_05"}, {"end": 3571.749, "text": "All right, so you can cover all these variable differences.", "start": 3568.368, "speaker": "SPEAKER_05"}, {"end": 3584.253, "text": "The third reason why we're down to a single name is to avoid the next input field, which says name on credit card or name on payment method, which then you have to ask yet again, because now you split the name, so you need the name that shows up on the credit.", "start": 3571.749, "speaker": "SPEAKER_05"}, {"end": 3586.134, "text": "Here, you don't have to ask it again.", "start": 3584.253, "speaker": "SPEAKER_05"}, {"end": 3588.895, "text": "You just ask for their name.", "start": 3586.134, "speaker": "SPEAKER_05"}, {"end": 3590.735, "text": "So we got their name.", "start": 3588.895, "speaker": "SPEAKER_05"}, {"end": 3591.916, "text": "Yay.", "start": 3590.735, "speaker": "SPEAKER_05"}, {"end": 3593.476, "text": "What are we going to do about shipping address?", "start": 3591.916, "speaker": "SPEAKER_05"}, {"end": 3597.673, "text": "We were able to collect a name.", "start": 3596.333, "speaker": "SPEAKER_05"}, {"end": 3599.194, "text": "That's good.", "start": 3597.673, "speaker": "SPEAKER_05"}, {"end": 3605.515, "text": "For here, what we're actually going to use is a principle called inline validation, which is helping you as you type.", "start": 3599.194, "speaker": "SPEAKER_05"}, {"end": 3610.476, "text": "It's real-time feedback that helps you avoid errors, helps you get to correct answers.", "start": 3605.515, "speaker": "SPEAKER_05"}, {"end": 3616.598, "text": "And so what Staples has done in this case is made use of the Google Maps autocomplete address API.", "start": 3610.476, "speaker": "SPEAKER_05"}, {"end": 3621.459, "text": "And the really neat thing about this API is you can restrict it down to particular countries or not.", "start": 3616.598, "speaker": "SPEAKER_05"}, {"end": 3635.927, "text": "Basically, you start typing, a dropdown with a bunch of autocomplete suggestion comes up, and in pretty much all cases I've ever seen, the address is there before you finish typing it.", "start": 3623.337, "speaker": "SPEAKER_05"}, {"end": 3644.774, "text": "So you save a bunch of time, you improve accuracy, you minimize errors, and just by typing 712 second, you hit that button, gone.", "start": 3635.927, "speaker": "SPEAKER_05"}, {"end": 3648.477, "text": "Uses geocation, helps you manage all this good stuff.", "start": 3644.774, "speaker": "SPEAKER_05"}, {"end": 3657.154, "text": "So here, through inline validation, we've now collected a shipping address, and we've done our best to make sure it is a valid shipping address.", "start": 3649.648, "speaker": "SPEAKER_05"}, {"end": 3662.439, "text": "And we haven't had people jump between 20 different fields to do so.", "start": 3657.154, "speaker": "SPEAKER_05"}, {"end": 3663.92, "text": "Saving effort like this pays off.", "start": 3662.439, "speaker": "SPEAKER_05"}, {"end": 3670.445, "text": "We did some tests not that long ago, comparing collecting city, state, and zip.", "start": 3663.92, "speaker": "SPEAKER_05"}, {"end": 3677.111, "text": "Now, granted, this is in US addresses, so different regions have different variants of what you can look up and how you can look it up in addresses.", "start": 3670.445, "speaker": "SPEAKER_05"}, {"end": 3679.699, "text": "and different structures.", "start": 3678.197, "speaker": "SPEAKER_05"}, {"end": 3682.703, "text": "Again, we can talk about that more deeply in Q&A if you want.", "start": 3679.699, "speaker": "SPEAKER_05"}, {"end": 3687.951, "text": "But here we were collecting city, and then collecting state, and then having people enter their zip code.", "start": 3682.703, "speaker": "SPEAKER_05"}, {"end": 3694.921, "text": "Whereas in this version, you entered your zip code, and we would automatically go and fetch the city and state based off of the zip code that you put in.", "start": 3687.951, "speaker": "SPEAKER_05"}, {"end": 3698.92, "text": "which was cool, so we infer that automatically.", "start": 3696.499, "speaker": "SPEAKER_05"}, {"end": 3708.962, "text": "And when we compared the two experiences, we found doing those three fields, city, state, and zip, took about 21 seconds, and this took about eight seconds.", "start": 3698.92, "speaker": "SPEAKER_05"}, {"end": 3718.384, "text": "And looking at personal preference, the majority of people strongly preferred doing it this way as opposed to this way, because it saves them a bunch of time and effort.", "start": 3708.962, "speaker": "SPEAKER_05"}, {"end": 3724.225, "text": "So taking this sort of principle and applying it to the whole address field helps a lot as well.", "start": 3718.384, "speaker": "SPEAKER_05"}, {"end": 3731.202, "text": "Now, we can use a similar technique for capturing email or contact information.", "start": 3725.599, "speaker": "SPEAKER_05"}, {"end": 3733.344, "text": "And so again, we'll use inline validation here.", "start": 3731.202, "speaker": "SPEAKER_05"}, {"end": 3739.507, "text": "And as you start typing your email address, we'll do an autocomplete based off of common email addresses.", "start": 3733.344, "speaker": "SPEAKER_05"}, {"end": 3743.469, "text": "And you can, again, tap on the thing that is right for you.", "start": 3739.507, "speaker": "SPEAKER_05"}, {"end": 3745.671, "text": "And you're off to the races.", "start": 3743.469, "speaker": "SPEAKER_05"}, {"end": 3746.611, "text": "Why do we have this?", "start": 3745.671, "speaker": "SPEAKER_05"}, {"end": 3749.453, "text": "Has anybody seen this repeat email address thing out there?", "start": 3746.611, "speaker": "SPEAKER_05"}, {"end": 3752.038, "text": "You've probably encountered this.", "start": 3751.017, "speaker": "SPEAKER_05"}, {"end": 3753.799, "text": "Why the hell do people do this to us?", "start": 3752.038, "speaker": "SPEAKER_05"}, {"end": 3754.46, "text": "It's cruel.", "start": 3753.799, "speaker": "SPEAKER_05"}, {"end": 3755.2, "text": "It's unusual.", "start": 3754.46, "speaker": "SPEAKER_05"}, {"end": 3756.461, "text": "It's punishment.", "start": 3755.2, "speaker": "SPEAKER_05"}, {"end": 3761.105, "text": "The reason why they do that is to get rid of this thing called hard bounces.", "start": 3756.461, "speaker": "SPEAKER_05"}, {"end": 3773.373, "text": "So on average, 15% of times you collect an email address, it will hard bounce, meaning it's not a valid email address, which is not what you want when you're setting up a new customer relationship.", "start": 3761.105, "speaker": "SPEAKER_05"}, {"end": 3776.476, "text": "So you ask twice to make sure that errors get cut down, what have you.", "start": 3773.373, "speaker": "SPEAKER_05"}, {"end": 3781.813, "text": "Our friend inline validation can get rid of this really erroneous stuff.", "start": 3777.392, "speaker": "SPEAKER_05"}, {"end": 3788.055, "text": "So when you type something that looks kind of like a Gmail account, but they mistyped it, you say, did you mean that?", "start": 3781.813, "speaker": "SPEAKER_05"}, {"end": 3788.755, "text": "You add it here.", "start": 3788.055, "speaker": "SPEAKER_05"}, {"end": 3793.117, "text": "This sort of interaction will cut down on hard bounces by a huge amount.", "start": 3788.755, "speaker": "SPEAKER_05"}, {"end": 3798.858, "text": "You can never get rid of the people who type joe at fart.com because they hate you.", "start": 3793.117, "speaker": "SPEAKER_05"}, {"end": 3801.479, "text": "Actually, what they hate you for is making them register again.", "start": 3798.858, "speaker": "SPEAKER_05"}, {"end": 3806.311, "text": "I had a real funny story for half a second.", "start": 3804.31, "speaker": "SPEAKER_05"}, {"end": 3807.611, "text": "I did this two days ago.", "start": 3806.311, "speaker": "SPEAKER_05"}, {"end": 3810.972, "text": "I had to register to download a white paper.", "start": 3807.611, "speaker": "SPEAKER_05"}, {"end": 3813.393, "text": "And I put in bob at fu.com.", "start": 3810.972, "speaker": "SPEAKER_05"}, {"end": 3820.475, "text": "And they're like, that email address has already been taken.", "start": 3813.393, "speaker": "SPEAKER_05"}, {"end": 3820.995, "text": "True story.", "start": 3820.475, "speaker": "SPEAKER_05"}, {"end": 3828.818, "text": "So yeah, verifying email address with a real-time message will get rid of some of those pesky balances.", "start": 3823.673, "speaker": "SPEAKER_05"}, {"end": 3829.719, "text": "Cool.", "start": 3828.818, "speaker": "SPEAKER_05"}, {"end": 3833.043, "text": "So now we've got name, we've got address, we've got email.", "start": 3829.719, "speaker": "SPEAKER_05"}, {"end": 3835.866, "text": "Last but not least, we want to collect payment info.", "start": 3833.043, "speaker": "SPEAKER_05"}, {"end": 3837.307, "text": "And there's a couple things happening here.", "start": 3835.866, "speaker": "SPEAKER_05"}, {"end": 3839.189, "text": "First, I'll show you the input mask pattern.", "start": 3837.307, "speaker": "SPEAKER_05"}, {"end": 3844.532, "text": "And then I'll just sort of reference, they also enabled the ability to scan a credit card.", "start": 3839.95, "speaker": "SPEAKER_05"}, {"end": 3847.914, "text": "So this is kind of available now, mobile, where you can just take a picture of your credit card.", "start": 3844.532, "speaker": "SPEAKER_05"}, {"end": 3851.656, "text": "I have no stats on how much people actually do or don't use this.", "start": 3847.914, "speaker": "SPEAKER_05"}, {"end": 3856.698, "text": "But it's an example of an attempt to try and use the sensors on the device to make this process easier.", "start": 3851.656, "speaker": "SPEAKER_05"}, {"end": 3862.14, "text": "You just point your phone at a credit card, it collects all the info, and you're done.", "start": 3856.698, "speaker": "SPEAKER_05"}, {"end": 3864.702, "text": "Pretty neat.", "start": 3862.14, "speaker": "SPEAKER_05"}, {"end": 3867.363, "text": "But let's look at the input mask implementation.", "start": 3864.702, "speaker": "SPEAKER_05"}, {"end": 3884.205, "text": "And the way this works is, note you're up with a big dial pad, so you're just collecting a bunch of numbers, you start typing your credit card number in, it highlights the type of credit card you have, because there's a pattern behind these, you don't have to select the type of credit card you have, so we can get rid of that input field.", "start": 3868.733, "speaker": "SPEAKER_05"}, {"end": 3899.571, "text": "We highlight that, and then what we're going to do is scoot this thing over, and now since you still have the numeric key, you just type in your expiration date, and then based on the credit card type you have, we change the icon to show you where your CVV security code is.", "start": 3884.885, "speaker": "SPEAKER_05"}, {"end": 3905.834, "text": "You type that, and without ever leaving the dial pad, you've entered all the information you need to process a credit card.", "start": 3899.571, "speaker": "SPEAKER_05"}, {"end": 3911.664, "text": "We've done a bunch of testing on this, and this format is actually very successful.", "start": 3907.562, "speaker": "SPEAKER_05"}, {"end": 3917.386, "text": "We tried it out on Android and iOS, and it works.", "start": 3911.664, "speaker": "SPEAKER_05"}, {"end": 3922.108, "text": "So last but not least, we've got name, address, contact info, credit card.", "start": 3917.386, "speaker": "SPEAKER_05"}, {"end": 3926.59, "text": "And then we have a smart default set here, which is use this address as a billing address.", "start": 3922.108, "speaker": "SPEAKER_05"}, {"end": 3929.971, "text": "You can undo that and change it if you want, but in many cases, you don't.", "start": 3926.59, "speaker": "SPEAKER_05"}, {"end": 3933.632, "text": "Here's how much it's going to cost, shipping, tax, everything you need.", "start": 3929.971, "speaker": "SPEAKER_05"}, {"end": 3934.573, "text": "Hit the button.", "start": 3933.632, "speaker": "SPEAKER_05"}, {"end": 3934.893, "text": "You're done.", "start": 3934.573, "speaker": "SPEAKER_05"}, {"end": 3945.009, "text": "All right, five input fields, done.", "start": 3937.138, "speaker": "SPEAKER_05"}, {"end": 3949.254, "text": "And again, we see a lot of iteration on what is a very important issue for a retailer.", "start": 3945.009, "speaker": "SPEAKER_05"}, {"end": 3951.417, "text": "Check out, huge important issue for a retailer.", "start": 3949.254, "speaker": "SPEAKER_05"}, {"end": 3963.531, "text": "We saw them reducing inputs, doing address lookup, trying inline validation, input masks, scanning credit cards, using smart defaults, all sorts of techniques.", "start": 3952.264, "speaker": "SPEAKER_05"}, {"end": 3965.412, "text": "And the impact of this was actually pretty dramatic.", "start": 3963.531, "speaker": "SPEAKER_05"}, {"end": 3976.959, "text": "They had a 50% reduction in checkout time, 1,000 basis point improvement in people who started checkout, engagement went up 40%, 70% reduction in clicks, all sorts of good stuff.", "start": 3965.412, "speaker": "SPEAKER_05"}, {"end": 3978.86, "text": "And there's a lot of big impact from this.", "start": 3976.959, "speaker": "SPEAKER_05"}, {"end": 3982.109, "text": "But again, I ask the same question.", "start": 3979.726, "speaker": "SPEAKER_05"}, {"end": 3985.472, "text": "Are we still just sort of moving pixels around?", "start": 3982.109, "speaker": "SPEAKER_05"}, {"end": 3987.755, "text": "Are we optimizing for a local maximum?", "start": 3985.472, "speaker": "SPEAKER_05"}, {"end": 3991.859, "text": "Or are we really moving towards the future that mobile enables?", "start": 3987.755, "speaker": "SPEAKER_05"}, {"end": 3992.96, "text": "And again, I'll say the same thing.", "start": 3991.859, "speaker": "SPEAKER_05"}, {"end": 3995.463, "text": "I think there is a lot of value to doing this.", "start": 3992.96, "speaker": "SPEAKER_05"}, {"end": 3999.146, "text": "If you go do this for your checkout flows, probably good things will happen to you.", "start": 3995.463, "speaker": "SPEAKER_05"}, {"end": 4001.563, "text": "A lot of improvements to be made.", "start": 4000.063, "speaker": "SPEAKER_05"}, {"end": 4005.364, "text": "And hopefully just showing you these kinds of improvements gets ideas going in your head.", "start": 4001.563, "speaker": "SPEAKER_05"}, {"end": 4006.044, "text": "Oh, we could do this.", "start": 4005.364, "speaker": "SPEAKER_05"}, {"end": 4006.644, "text": "We could do that.", "start": 4006.044, "speaker": "SPEAKER_05"}, {"end": 4007.765, "text": "Why don't we try this and try it?", "start": 4006.644, "speaker": "SPEAKER_05"}, {"end": 4008.525, "text": "Great.", "start": 4007.765, "speaker": "SPEAKER_05"}, {"end": 4016.366, "text": "All sorts of optimizations to do and clearly examples out there of those kinds of optimizations moving the needle in the right way.", "start": 4008.525, "speaker": "SPEAKER_05"}, {"end": 4025.488, "text": "Going from 22 input field form down to a five field input form that reduces errors dramatically, that seems like a win to me.", "start": 4016.366, "speaker": "SPEAKER_05"}, {"end": 4031.483, "text": "But let's go back to Amazon for a second and check out how their payment process works on mobile.", "start": 4026.578, "speaker": "SPEAKER_05"}, {"end": 4036.507, "text": "So here I see a lovely little live pet that I would like.", "start": 4031.483, "speaker": "SPEAKER_05"}, {"end": 4038.449, "text": "And so I will hit buy now.", "start": 4036.507, "speaker": "SPEAKER_05"}, {"end": 4043.254, "text": "And then the only thing I do is I just swipe.", "start": 4038.449, "speaker": "SPEAKER_05"}, {"end": 4045.035, "text": "Done.", "start": 4043.254, "speaker": "SPEAKER_05"}, {"end": 4046.777, "text": "It's going to be at my house tomorrow.", "start": 4045.035, "speaker": "SPEAKER_05"}, {"end": 4051.361, "text": "That's pretty awesome, right?", "start": 4049.579, "speaker": "SPEAKER_05"}, {"end": 4059.288, "text": "I just hit buy now, and then I just go swipe, and the little lovely pet is at my house tomorrow.", "start": 4051.361, "speaker": "SPEAKER_05"}, {"end": 4060.409, "text": "Super fast process.", "start": 4059.288, "speaker": "SPEAKER_05"}, {"end": 4076.024, "text": "Again, to echo some of the talk around psychology that I think kicked off the conference yesterday, reducing effort, reducing cognitive load, making it frictionless, fast, and easy for people to buy things.", "start": 4063.618, "speaker": "SPEAKER_05"}, {"end": 4082.548, "text": "There's a reason Amazon is doing so well, because when you see something, you just got to go, and you paid for it.", "start": 4076.024, "speaker": "SPEAKER_05"}, {"end": 4092.644, "text": "Now, granted, they had to collect this upfront information first, but maybe you could use that pattern I just showed you a second ago from Staples to do that one time.", "start": 4083.318, "speaker": "SPEAKER_05"}, {"end": 4094.925, "text": "So that's pretty painless to get at one time.", "start": 4092.644, "speaker": "SPEAKER_05"}, {"end": 4097.647, "text": "And then after that, you start thinking more like this.", "start": 4094.925, "speaker": "SPEAKER_05"}, {"end": 4104.391, "text": "And not only is this process super fast, but it actually aligns with how people actually use mobile devices, which is specifically one-handed.", "start": 4097.647, "speaker": "SPEAKER_05"}, {"end": 4106.693, "text": "There's a lot of one-handed use of devices.", "start": 4104.391, "speaker": "SPEAKER_05"}, {"end": 4117.213, "text": "And as devices become bigger, as our phones continue to become more fablity, things that are up high become more ouchity to use.", "start": 4107.813, "speaker": "SPEAKER_05"}, {"end": 4121.677, "text": "So this is sort of the comfortable, nice-to-hit area on your phone.", "start": 4118.136, "speaker": "SPEAKER_05"}, {"end": 4127.94, "text": "And so these interactions that are down in the bottom, like that swipe and tap, really works a long way.", "start": 4121.677, "speaker": "SPEAKER_05"}, {"end": 4129.981, "text": "Our fingers are only so long.", "start": 4127.94, "speaker": "SPEAKER_05"}, {"end": 4138.064, "text": "And when we start to build things that are designed to be used in our hands, we have to think about how far can we, especially one-handed, reach our fingers.", "start": 4129.981, "speaker": "SPEAKER_05"}, {"end": 4140.606, "text": "They only stretch so far.", "start": 4139.365, "speaker": "SPEAKER_05"}, {"end": 4152.456, "text": "And looking at this in terms of market trends, you can see that these big phones, these phablet-sized things, five to seven inches, are really increasing in share at the expense of these smaller-sized things.", "start": 4140.606, "speaker": "SPEAKER_05"}, {"end": 4164.047, "text": "So when we work on designs, we really want to make sure that we're thinking about these market trends, hoping that there's some longevity behind what we build, so we don't just, again, build for today, but we look at where the market is going.", "start": 4152.456, "speaker": "SPEAKER_05"}, {"end": 4170.528, "text": "Ani showed this payment request API that Google has been working on with web browsers.", "start": 4165.185, "speaker": "SPEAKER_05"}, {"end": 4180.074, "text": "And I really love the philosophy of this, because it illustrates kind of what I was talking about a second ago, which is you come over to, say, a merchant on Shopify.", "start": 4170.528, "speaker": "SPEAKER_05"}, {"end": 4186.177, "text": "You hit the Checkout button, upslides this little thing that has all of this information, because it was stored in the browser.", "start": 4180.074, "speaker": "SPEAKER_05"}, {"end": 4187.818, "text": "And you just hit Pay, and you're done.", "start": 4186.177, "speaker": "SPEAKER_05"}, {"end": 4192.098, "text": "All right, super frictionless, super fast, super easy.", "start": 4189.277, "speaker": "SPEAKER_05"}, {"end": 4194.659, "text": "Granted, browsers have to implement this.", "start": 4192.098, "speaker": "SPEAKER_05"}, {"end": 4200.542, "text": "And granted, people have to go and put their money credentials and off inside the browser.", "start": 4194.659, "speaker": "SPEAKER_05"}, {"end": 4210.806, "text": "But if it's there, wow, it's almost as fast and easy as that Amazon system by taking your pre-populated info and giving you this single action pop-up.", "start": 4200.542, "speaker": "SPEAKER_05"}, {"end": 4219.064, "text": "So once again, we see Amazon not sitting still in their implementation of this and continuing to iterate.", "start": 4211.802, "speaker": "SPEAKER_05"}, {"end": 4221.504, "text": "We saw a lot of iterations on the payment experience.", "start": 4219.064, "speaker": "SPEAKER_05"}, {"end": 4226.766, "text": "But once more, I want to ask, what are we going toward?", "start": 4221.504, "speaker": "SPEAKER_05"}, {"end": 4228.266, "text": "What is the bar here?", "start": 4226.766, "speaker": "SPEAKER_05"}, {"end": 4229.406, "text": "What is the North Star?", "start": 4228.266, "speaker": "SPEAKER_05"}, {"end": 4231.407, "text": "Where do we want to go with all this iteration?", "start": 4229.406, "speaker": "SPEAKER_05"}, {"end": 4234.227, "text": "One tap and a swipe, that's pretty damn good.", "start": 4231.407, "speaker": "SPEAKER_05"}, {"end": 4235.067, "text": "Can we go further?", "start": 4234.227, "speaker": "SPEAKER_05"}, {"end": 4237.068, "text": "Or can we take this?", "start": 4235.067, "speaker": "SPEAKER_05"}, {"end": 4240.87, "text": "And certainly, Amazon thinks we can go a lot further.", "start": 4238.248, "speaker": "SPEAKER_05"}, {"end": 4243.191, "text": "Let's look at some of the stuff that they have done recently.", "start": 4240.87, "speaker": "SPEAKER_05"}, {"end": 4245.353, "text": "This is the Amazon Dash button.", "start": 4243.191, "speaker": "SPEAKER_05"}, {"end": 4252.077, "text": "This is a little thing that has Wi-Fi, it's connected to your account, and it's effectively a single-use device.", "start": 4245.353, "speaker": "SPEAKER_05"}, {"end": 4252.978, "text": "It only does one thing.", "start": 4252.077, "speaker": "SPEAKER_05"}, {"end": 4256.2, "text": "You know what this thing does?", "start": 4252.978, "speaker": "SPEAKER_05"}, {"end": 4258.421, "text": "Orders Tide for you.", "start": 4256.2, "speaker": "SPEAKER_05"}, {"end": 4261.123, "text": "So you tap it one time, laundry detergent shows up.", "start": 4258.421, "speaker": "SPEAKER_05"}, {"end": 4264.037, "text": "That's it.", "start": 4262.376, "speaker": "SPEAKER_05"}, {"end": 4267.858, "text": "And in context, it's actually pretty darn useful.", "start": 4264.037, "speaker": "SPEAKER_05"}, {"end": 4269.219, "text": "You put it on the washing machine.", "start": 4267.858, "speaker": "SPEAKER_05"}, {"end": 4274.682, "text": "Where do you run out of washing machine detergent?", "start": 4269.219, "speaker": "SPEAKER_05"}, {"end": 4277.403, "text": "In the freaking laundry room, right?", "start": 4274.682, "speaker": "SPEAKER_05"}, {"end": 4279.804, "text": "You don't think, oh yeah, I need more laundry.", "start": 4277.403, "speaker": "SPEAKER_05"}, {"end": 4285.426, "text": "Maybe it pops into your brain somewhere, but right here and there at that moment is when you need the stuff.", "start": 4279.804, "speaker": "SPEAKER_05"}, {"end": 4286.827, "text": "So you tap it once and you're done.", "start": 4285.426, "speaker": "SPEAKER_05"}, {"end": 4295.556, "text": "There's no browsing, no opening an app, no finding the app, no clicking over to search.", "start": 4289.612, "speaker": "SPEAKER_05"}, {"end": 4298.077, "text": "I mean, think about all the steps this removes, right?", "start": 4295.556, "speaker": "SPEAKER_05"}, {"end": 4299.398, "text": "You just go like this.", "start": 4298.077, "speaker": "SPEAKER_05"}, {"end": 4300.839, "text": "OK, it's coming.", "start": 4299.398, "speaker": "SPEAKER_05"}, {"end": 4303.26, "text": "Versus, oh, let me unlock my phone.", "start": 4300.839, "speaker": "SPEAKER_05"}, {"end": 4304.361, "text": "Let me hit the app.", "start": 4303.26, "speaker": "SPEAKER_05"}, {"end": 4305.702, "text": "Let me do a search for Tide.", "start": 4304.361, "speaker": "SPEAKER_05"}, {"end": 4306.923, "text": "Oh, is that the Tide I like?", "start": 4305.702, "speaker": "SPEAKER_05"}, {"end": 4307.343, "text": "I don't know.", "start": 4306.923, "speaker": "SPEAKER_05"}, {"end": 4308.464, "text": "Is this the Tide I like?", "start": 4307.343, "speaker": "SPEAKER_05"}, {"end": 4309.284, "text": "Oh, OK.", "start": 4308.464, "speaker": "SPEAKER_05"}, {"end": 4309.925, "text": "Now let me do it.", "start": 4309.284, "speaker": "SPEAKER_05"}, {"end": 4311.626, "text": "And then I get my buy and swipe.", "start": 4309.925, "speaker": "SPEAKER_05"}, {"end": 4314.783, "text": "But this eliminates all of that.", "start": 4312.621, "speaker": "SPEAKER_05"}, {"end": 4317.825, "text": "Press it once, and detergent is on its way.", "start": 4314.783, "speaker": "SPEAKER_05"}, {"end": 4320.087, "text": "In one tap, in context.", "start": 4317.825, "speaker": "SPEAKER_05"}, {"end": 4326.012, "text": "And this is what I promised to be one of those weird examples, because you're like, I can't do that for my insurance thing.", "start": 4320.087, "speaker": "SPEAKER_05"}, {"end": 4328.214, "text": "Maybe not.", "start": 4326.012, "speaker": "SPEAKER_05"}, {"end": 4330.055, "text": "I know the insurance question is going to come up later.", "start": 4328.214, "speaker": "SPEAKER_05"}, {"end": 4337.281, "text": "How can I do this with insurance?", "start": 4330.055, "speaker": "SPEAKER_05"}, {"end": 4337.841, "text": "Thank you.", "start": 4337.281, "speaker": "SPEAKER_05"}, {"end": 4339.903, "text": "You have too much money already in insurance.", "start": 4337.841, "speaker": "SPEAKER_05"}, {"end": 4340.844, "text": "You don't need to do anything.", "start": 4339.903, "speaker": "SPEAKER_05"}, {"end": 4344.25, "text": "Just keep doing what you're doing.", "start": 4341.989, "speaker": "SPEAKER_05"}, {"end": 4346.11, "text": "You have all the money you could ever want.", "start": 4344.25, "speaker": "SPEAKER_05"}, {"end": 4346.75, "text": "Don't be greedy.", "start": 4346.11, "speaker": "SPEAKER_05"}, {"end": 4350.651, "text": "I'm serious.", "start": 4346.75, "speaker": "SPEAKER_05"}, {"end": 4353.832, "text": "Insurance is a lot of money.", "start": 4350.651, "speaker": "SPEAKER_05"}, {"end": 4359.354, "text": "But I don't mean necessarily go and implement an Amazon Dash button for your thing.", "start": 4353.832, "speaker": "SPEAKER_05"}, {"end": 4361.974, "text": "But have this kind of vision in your head.", "start": 4359.354, "speaker": "SPEAKER_05"}, {"end": 4364.435, "text": "How far can we take it?", "start": 4361.974, "speaker": "SPEAKER_05"}, {"end": 4367.276, "text": "What is the North Star we should go for?", "start": 4364.435, "speaker": "SPEAKER_05"}, {"end": 4371.626, "text": "Let's look at that in another example.", "start": 4369.385, "speaker": "SPEAKER_05"}, {"end": 4376.087, "text": "Amazon has this new line of physical stores called Amazon Go.", "start": 4371.626, "speaker": "SPEAKER_05"}, {"end": 4379.128, "text": "They boast no lines, no checkout.", "start": 4376.087, "speaker": "SPEAKER_05"}, {"end": 4381.889, "text": "To me, this is an example of an awesome North Star.", "start": 4379.128, "speaker": "SPEAKER_05"}, {"end": 4386.37, "text": "Instead of let's optimize checkout, how about no checkout?", "start": 4381.889, "speaker": "SPEAKER_05"}, {"end": 4387.33, "text": "I love that.", "start": 4386.37, "speaker": "SPEAKER_05"}, {"end": 4389.131, "text": "Our goal is to have no checkout.", "start": 4387.33, "speaker": "SPEAKER_05"}, {"end": 4396.411, "text": "And if you have that kind of vision in your head, no checkout, then you can see how you might start behaving differently.", "start": 4390.646, "speaker": "SPEAKER_05"}, {"end": 4409.22, "text": "You could still go through the iteration process, but as you're iterating, you're constantly trying to get to this idealized end state, something that goes beyond what you have today, something that feels like the future or is the future.", "start": 4396.411, "speaker": "SPEAKER_05"}, {"end": 4412.283, "text": "So let's look at this for half a second.", "start": 4409.22, "speaker": "SPEAKER_05"}, {"end": 4415.705, "text": "Here's how Amazon works from a user perspective.", "start": 4412.283, "speaker": "SPEAKER_05"}, {"end": 4418.328, "text": "And I apologize for the marketing-y music thing.", "start": 4415.705, "speaker": "SPEAKER_05"}, {"end": 4420.109, "text": "I'll come back and talk after it.", "start": 4418.328, "speaker": "SPEAKER_05"}, {"end": 4433.296, "text": "Four years ago, we started to wonder, what would shopping look like if you could walk into a store, grab what you want, and just go?", "start": 4423.509, "speaker": "SPEAKER_00"}, {"end": 4434.476, "text": "Thief!", "start": 4433.296, "speaker": "SPEAKER_00"}, {"end": 4443.202, "text": "What if we could weave the most advanced machine learning, computer vision, and AI into the very fabric of a store, so you never have to wait in line?", "start": 4434.476, "speaker": "SPEAKER_00"}, {"end": 4449.046, "text": "No lines, no checkouts, no registers.", "start": 4443.202, "speaker": "SPEAKER_00"}, {"end": 4450.988, "text": "Welcome to Amazon Go.", "start": 4449.046, "speaker": "SPEAKER_00"}, {"end": 4455.984, "text": "OK, so how does it work?", "start": 4454.902, "speaker": "SPEAKER_05"}, {"end": 4462.212, "text": "You just walk in, you tap your phone on the thing, you take what you want, and you leave.", "start": 4455.984, "speaker": "SPEAKER_05"}, {"end": 4464.616, "text": "They call it walk-out shopping, right?", "start": 4462.212, "speaker": "SPEAKER_05"}, {"end": 4468.281, "text": "And it's an attempt to create a frictionless shopping experience.", "start": 4464.616, "speaker": "SPEAKER_05"}, {"end": 4476.837, "text": "Now what's really going on behind the scenes, because that's a whole other story, machine learning, facial recognition, microphones.", "start": 4469.955, "speaker": "SPEAKER_05"}, {"end": 4478.177, "text": "Let's look at this for half a second.", "start": 4476.837, "speaker": "SPEAKER_05"}, {"end": 4481.578, "text": "So how does Amazon Go actually work?", "start": 4478.177, "speaker": "SPEAKER_05"}, {"end": 4486.399, "text": "So when you come in, you have a little app on your phone that you scan with the sensor.", "start": 4481.578, "speaker": "SPEAKER_05"}, {"end": 4488.8, "text": "It identifies that user.", "start": 4486.399, "speaker": "SPEAKER_05"}, {"end": 4489.52, "text": "There's a camera.", "start": 4488.8, "speaker": "SPEAKER_05"}, {"end": 4498.035, "text": "And the camera will actually use facial recognition to uniquely identify each shopper.", "start": 4491.792, "speaker": "SPEAKER_05"}, {"end": 4500.956, "text": "And then throughout the store, this is the creepy part, right?", "start": 4498.035, "speaker": "SPEAKER_05"}, {"end": 4506.158, "text": "Cameras see what you do, and whether that item stays with you or not.", "start": 4500.956, "speaker": "SPEAKER_05"}, {"end": 4512.121, "text": "There's images captured in possession of the item, we just talked about that.", "start": 4506.158, "speaker": "SPEAKER_05"}, {"end": 4517.263, "text": "There's microphones that know where you are and where people are, so they know where to kind of point the cameras.", "start": 4512.121, "speaker": "SPEAKER_05"}, {"end": 4520.665, "text": "There's infrared pressure and load sensors on all the different shelves.", "start": 4517.263, "speaker": "SPEAKER_05"}, {"end": 4526.951, "text": "Note when an item is picked up or picked back.", "start": 4521.525, "speaker": "SPEAKER_05"}, {"end": 4543.326, "text": "And then there's a whole bunch of kind of deep learning and or machine learning kind of systems that make sense of all this data behind the scenes to tell where people are, because literally what they're doing is they're tracking everything in the store in real time, including human beings.", "start": 4526.951, "speaker": "SPEAKER_05"}, {"end": 4550.25, "text": "So they have a map of, okay, this person's over here, these items are in flight, these items are with this person, this person left with these items, what have you.", "start": 4543.326, "speaker": "SPEAKER_05"}, {"end": 4551.971, "text": "A lot of complex stuff, right?", "start": 4550.25, "speaker": "SPEAKER_05"}, {"end": 4557.333, "text": "Facial recognition, microphones, cameras, machine learning, all this stuff behind the scenes.", "start": 4551.971, "speaker": "SPEAKER_05"}, {"end": 4566.179, "text": "And the interesting thing about this, in my opinion, is all this complicated stuff This technology is very complicated.", "start": 4557.333, "speaker": "SPEAKER_05"}, {"end": 4577.626, "text": "Think about all the things that they're doing in their track, and all of that stuff is going on from a hardware and from a software perspective to make a very simple and frictionless shopping experience.", "start": 4566.179, "speaker": "SPEAKER_05"}, {"end": 4586.031, "text": "And this is the trick in a lot of what we do in the technology industry, is take all this technical complexity, bring it on yourself.", "start": 4577.626, "speaker": "SPEAKER_05"}, {"end": 4589.033, "text": "Take it on as a burden, and you take it.", "start": 4586.031, "speaker": "SPEAKER_05"}, {"end": 4590.554, "text": "Don't put it on the human being.", "start": 4589.033, "speaker": "SPEAKER_05"}, {"end": 4594.322, "text": "You deal with it.", "start": 4593.221, "speaker": "SPEAKER_05"}, {"end": 4600.906, "text": "All the human being have to do is walk, take thing, and leave.", "start": 4594.322, "speaker": "SPEAKER_05"}, {"end": 4602.888, "text": "That's how you make a happy human.", "start": 4600.906, "speaker": "SPEAKER_05"}, {"end": 4607.391, "text": "You take all the benefits that they get from all that technology, and you throw it behind the scenes.", "start": 4602.888, "speaker": "SPEAKER_05"}, {"end": 4608.672, "text": "They don't even have to think about it.", "start": 4607.391, "speaker": "SPEAKER_05"}, {"end": 4609.973, "text": "They just go and take things.", "start": 4608.672, "speaker": "SPEAKER_05"}, {"end": 4611.053, "text": "They feel like they're stealing.", "start": 4609.973, "speaker": "SPEAKER_05"}, {"end": 4611.634, "text": "They're elated.", "start": 4611.053, "speaker": "SPEAKER_05"}, {"end": 4612.394, "text": "They're kind of nervous.", "start": 4611.634, "speaker": "SPEAKER_05"}, {"end": 4615.516, "text": "Oh, my god.", "start": 4612.394, "speaker": "SPEAKER_05"}, {"end": 4617.698, "text": "Whereas the machines do all the heavy lifting.", "start": 4615.516, "speaker": "SPEAKER_05"}, {"end": 4626.599, "text": "Again, why I like that example of Amazon Go is because I think it's such an illustrative example of this.", "start": 4621.797, "speaker": "SPEAKER_05"}, {"end": 4631.282, "text": "Look at all the stuff Amazon is doing to allow you just to walk in and walk out of a store.", "start": 4626.599, "speaker": "SPEAKER_05"}, {"end": 4633.423, "text": "No lines, no checkout, no anything.", "start": 4631.282, "speaker": "SPEAKER_05"}, {"end": 4638.851, "text": "Coming back to the Dash button, this is another great example of that philosophy.", "start": 4635.488, "speaker": "SPEAKER_05"}, {"end": 4641.452, "text": "This thing has Wi-Fi, it connects to your account.", "start": 4638.851, "speaker": "SPEAKER_05"}, {"end": 4645.715, "text": "That account is built onto a complicated billing system.", "start": 4641.452, "speaker": "SPEAKER_05"}, {"end": 4650.318, "text": "That billing system has returns, that billing system has processings associated with it.", "start": 4645.715, "speaker": "SPEAKER_05"}, {"end": 4659.405, "text": "It's even connected to a much deeper distribution and delivery network where there's warehouses around the entire world that do the distribution and they work with carriers.", "start": 4650.318, "speaker": "SPEAKER_05"}, {"end": 4663.988, "text": "I mean, there's a gazillion things happening behind this tiny little button when you push it.", "start": 4659.405, "speaker": "SPEAKER_05"}, {"end": 4666.73, "text": "What do you need to do?", "start": 4664.668, "speaker": "SPEAKER_05"}, {"end": 4668.912, "text": "Push the little button.", "start": 4666.73, "speaker": "SPEAKER_05"}, {"end": 4671.875, "text": "Everything else behind there is then working for you.", "start": 4668.912, "speaker": "SPEAKER_05"}, {"end": 4676.22, "text": "And it ultimately delivers this very frictionless and natural experience.", "start": 4671.875, "speaker": "SPEAKER_05"}, {"end": 4682.946, "text": "And again, when we want to be inspired by things, to me, this is kind of a nice North Star-y thing to dwell on.", "start": 4676.22, "speaker": "SPEAKER_05"}, {"end": 4686.029, "text": "To stick with this theme for a second, let's come back to Apple for a minute.", "start": 4682.946, "speaker": "SPEAKER_05"}, {"end": 4691.836, "text": "I think the new AirPods are a great example of this kind of moment as well.", "start": 4686.873, "speaker": "SPEAKER_05"}, {"end": 4698.259, "text": "So in order to connect your AirPods to any device, you just open the lid.", "start": 4691.836, "speaker": "SPEAKER_05"}, {"end": 4700.141, "text": "That's it.", "start": 4698.259, "speaker": "SPEAKER_05"}, {"end": 4701.601, "text": "You open the lid, a little thing pops up.", "start": 4700.141, "speaker": "SPEAKER_05"}, {"end": 4702.982, "text": "It says connecting, and they're connected.", "start": 4701.601, "speaker": "SPEAKER_05"}, {"end": 4704.783, "text": "You're done.", "start": 4702.982, "speaker": "SPEAKER_05"}, {"end": 4706.404, "text": "I didn't touch any button.", "start": 4704.783, "speaker": "SPEAKER_05"}, {"end": 4707.585, "text": "I didn't open any app.", "start": 4706.404, "speaker": "SPEAKER_05"}, {"end": 4709.026, "text": "I'm not hitting any UI, nothing.", "start": 4707.585, "speaker": "SPEAKER_05"}, {"end": 4712.808, "text": "I just flip open the lid on the case, and I connect it.", "start": 4709.026, "speaker": "SPEAKER_05"}, {"end": 4715.554, "text": "Now, you want to start playing music, what do you do?", "start": 4713.552, "speaker": "SPEAKER_05"}, {"end": 4717.536, "text": "You just put it in your ear.", "start": 4715.554, "speaker": "SPEAKER_05"}, {"end": 4718.998, "text": "It starts playing music.", "start": 4717.536, "speaker": "SPEAKER_05"}, {"end": 4727.146, "text": "If you want the music to stop, you just take it out of the ear.", "start": 4718.998, "speaker": "SPEAKER_05"}, {"end": 4728.387, "text": "I know, you're all stunned.", "start": 4727.146, "speaker": "SPEAKER_05"}, {"end": 4729.649, "text": "Like, no, it can't be like that.", "start": 4728.387, "speaker": "SPEAKER_05"}, {"end": 4730.93, "text": "I got to hit a button, right?", "start": 4729.649, "speaker": "SPEAKER_05"}, {"end": 4733.492, "text": "It's got to be an app.", "start": 4730.93, "speaker": "SPEAKER_05"}, {"end": 4736.275, "text": "I got to sign into an app.", "start": 4733.492, "speaker": "SPEAKER_05"}, {"end": 4736.596, "text": "Nothing.", "start": 4736.275, "speaker": "SPEAKER_05"}, {"end": 4743.319, "text": "It's this frictionless, graphical user interface-less kind of natural interaction.", "start": 4737.596, "speaker": "SPEAKER_05"}, {"end": 4752.665, "text": "And this term natural user interaction, NUI, it's been batted around for a while, but the general concept is we create interfaces that are based on a lifetime of learning.", "start": 4743.319, "speaker": "SPEAKER_05"}, {"end": 4755.306, "text": "Amazon Go is a great example of a natural user interface.", "start": 4752.665, "speaker": "SPEAKER_05"}, {"end": 4758.288, "text": "You walk in, you pick something up, you walk out.", "start": 4755.306, "speaker": "SPEAKER_05"}, {"end": 4759.228, "text": "You know how to do that.", "start": 4758.288, "speaker": "SPEAKER_05"}, {"end": 4761.93, "text": "You've been doing that your whole life.", "start": 4759.228, "speaker": "SPEAKER_05"}, {"end": 4764.471, "text": "Right?", "start": 4761.93, "speaker": "SPEAKER_05"}, {"end": 4766.533, "text": "So here, you want music to start playing, you just put in, done.", "start": 4764.471, "speaker": "SPEAKER_05"}, {"end": 4770.781, "text": "Now, if you want to charge these things, you just put them in the case.", "start": 4768.221, "speaker": "SPEAKER_05"}, {"end": 4775.742, "text": "Case charges them.", "start": 4770.781, "speaker": "SPEAKER_05"}, {"end": 4777.743, "text": "Pretty darn cool, right?", "start": 4775.742, "speaker": "SPEAKER_05"}, {"end": 4781.163, "text": "Another example of a natural user interface we saw at this conference.", "start": 4777.743, "speaker": "SPEAKER_05"}, {"end": 4783.404, "text": "Google Home, just talk to it.", "start": 4781.163, "speaker": "SPEAKER_05"}, {"end": 4786.224, "text": "It'll give you answers to things, help you manage your text.", "start": 4783.404, "speaker": "SPEAKER_05"}, {"end": 4787.224, "text": "You know how to talk.", "start": 4786.224, "speaker": "SPEAKER_05"}, {"end": 4788.284, "text": "Most of us know how to talk.", "start": 4787.224, "speaker": "SPEAKER_05"}, {"end": 4790.845, "text": "Some of us have some problems talking.", "start": 4788.284, "speaker": "SPEAKER_05"}, {"end": 4795.886, "text": "But we know how to talk.", "start": 4790.845, "speaker": "SPEAKER_05"}, {"end": 4797.006, "text": "So see a pattern here?", "start": 4795.886, "speaker": "SPEAKER_05"}, {"end": 4802.551, "text": "between the Dash and the AirPods and the Google Home.", "start": 4798.59, "speaker": "SPEAKER_05"}, {"end": 4812.035, "text": "I think the pattern here is this differentiation between these natural user interface type experiences that we know how to use based on a lifetime of living.", "start": 4802.551, "speaker": "SPEAKER_05"}, {"end": 4816.917, "text": "They're frictionless, and I think these things serve as a really great north star for where we can go.", "start": 4812.035, "speaker": "SPEAKER_05"}, {"end": 4821.9, "text": "Can we design checkout with the goal of no checkout?", "start": 4817.817, "speaker": "SPEAKER_05"}, {"end": 4825.102, "text": "Can we design sign-in with the goal of no sign-in?", "start": 4821.9, "speaker": "SPEAKER_05"}, {"end": 4830.986, "text": "It's these kind of aspirations where I think we can do more than just incremental improvements.", "start": 4825.102, "speaker": "SPEAKER_05"}, {"end": 4839.832, "text": "Because while, again, I'm going to jump back to this picture that I showed earlier around sign-in.", "start": 4830.986, "speaker": "SPEAKER_05"}, {"end": 4841.994, "text": "This is what I'm really interested in.", "start": 4839.832, "speaker": "SPEAKER_05"}, {"end": 4846.317, "text": "What's that moment where we just change the situation?", "start": 4841.994, "speaker": "SPEAKER_05"}, {"end": 4851.632, "text": "Where do you go from this ongoing iteration, which I'll say again, has value.", "start": 4847.429, "speaker": "SPEAKER_05"}, {"end": 4852.813, "text": "I'm not poo-pooing this, right?", "start": 4851.632, "speaker": "SPEAKER_05"}, {"end": 4854.174, "text": "There's a lot of value in this.", "start": 4852.813, "speaker": "SPEAKER_05"}, {"end": 4858.777, "text": "But what's the moment where the bit flips and the dynamics change?", "start": 4854.174, "speaker": "SPEAKER_05"}, {"end": 4862.961, "text": "And all of a sudden, registration is just putting your finger on something.", "start": 4858.777, "speaker": "SPEAKER_05"}, {"end": 4866.823, "text": "Or all of a sudden, buying something is just walking out of a store.", "start": 4862.961, "speaker": "SPEAKER_05"}, {"end": 4868.885, "text": "I think those moments are really, really interesting.", "start": 4866.823, "speaker": "SPEAKER_05"}, {"end": 4881.512, "text": "And just being inspired by those goes a really long way, hence why And I keep showing these natural user interfaces because I think they're a great North Star from an inspiration perspective.", "start": 4868.885, "speaker": "SPEAKER_05"}, {"end": 4888.553, "text": "And coming back to these for half a second, what better example of a natural user interface than using your eyes to see?", "start": 4881.512, "speaker": "SPEAKER_05"}, {"end": 4896.715, "text": "And so this new thing from Snapchat, Spectacles, make use of a camera on these lenses to give you 180 degree of what you're seeing.", "start": 4888.553, "speaker": "SPEAKER_05"}, {"end": 4898.156, "text": "It's actually pretty cool.", "start": 4896.715, "speaker": "SPEAKER_05"}, {"end": 4901.096, "text": "You can just turn on recording by just tapping the glasses.", "start": 4898.156, "speaker": "SPEAKER_05"}, {"end": 4902.297, "text": "That's it.", "start": 4901.096, "speaker": "SPEAKER_05"}, {"end": 4906.946, "text": "But then, This hardware experience is just so simple.", "start": 4902.297, "speaker": "SPEAKER_05"}, {"end": 4909.166, "text": "You wear glasses, you tap them, it records what you see.", "start": 4906.946, "speaker": "SPEAKER_05"}, {"end": 4911.087, "text": "Wow, super easy.", "start": 4909.166, "speaker": "SPEAKER_05"}, {"end": 4913.908, "text": "The software experience is almost the exact opposite.", "start": 4911.087, "speaker": "SPEAKER_05"}, {"end": 4922.49, "text": "And I had this moment hit me when I was using these things and I tried to get the video off of them and I felt like I went back.", "start": 4913.908, "speaker": "SPEAKER_05"}, {"end": 4925.663, "text": "in time, 10 years.", "start": 4923.761, "speaker": "SPEAKER_05"}, {"end": 4931.408, "text": "Because to use these things, to get the video from them and send them to someone, I have to take out my phone.", "start": 4925.663, "speaker": "SPEAKER_05"}, {"end": 4932.689, "text": "I have to tap on an app.", "start": 4931.408, "speaker": "SPEAKER_05"}, {"end": 4933.97, "text": "I have to open the app.", "start": 4932.689, "speaker": "SPEAKER_05"}, {"end": 4936.512, "text": "I have to go and tap on my memories.", "start": 4933.97, "speaker": "SPEAKER_05"}, {"end": 4939.114, "text": "And in my memories, I have to find the thing I recorded.", "start": 4936.512, "speaker": "SPEAKER_05"}, {"end": 4943.638, "text": "And then I have to go and flip through these things and tap multiple times to find the video.", "start": 4939.114, "speaker": "SPEAKER_05"}, {"end": 4946.36, "text": "And then when I find the video, I have to tap on Send.", "start": 4943.638, "speaker": "SPEAKER_05"}, {"end": 4949.363, "text": "And then after I tap on Send, I have to pick who I'm going to send it to.", "start": 4946.36, "speaker": "SPEAKER_05"}, {"end": 4950.484, "text": "And then I can actually send it.", "start": 4949.363, "speaker": "SPEAKER_05"}, {"end": 4958.838, "text": "Remember how I said we want to hide complexity from the user experience?", "start": 4953.694, "speaker": "SPEAKER_05"}, {"end": 4962.862, "text": "This app open and UI dance is not a great example of that.", "start": 4958.838, "speaker": "SPEAKER_05"}, {"end": 4968.146, "text": "And many times what we do with our software is we actually show that complexity to people.", "start": 4962.862, "speaker": "SPEAKER_05"}, {"end": 4972.409, "text": "In the Amazon Go example, we're trying to bury all that complexity, get it out of there.", "start": 4968.146, "speaker": "SPEAKER_05"}, {"end": 4973.83, "text": "Don't make people worry about it.", "start": 4972.409, "speaker": "SPEAKER_05"}, {"end": 4977.793, "text": "But what we do with software a lot of times is we shove it in people's faces.", "start": 4973.83, "speaker": "SPEAKER_05"}, {"end": 4978.834, "text": "And why?", "start": 4977.793, "speaker": "SPEAKER_05"}, {"end": 4980.135, "text": "Because we can't make a decision.", "start": 4978.834, "speaker": "SPEAKER_05"}, {"end": 4983.971, "text": "We can't decide what matters most right now.", "start": 4981.349, "speaker": "SPEAKER_05"}, {"end": 4986.552, "text": "And so what we do is we just throw every option at people.", "start": 4983.971, "speaker": "SPEAKER_05"}, {"end": 4988.534, "text": "Oh, I can't decide what matters at this point in time.", "start": 4986.552, "speaker": "SPEAKER_05"}, {"end": 4989.074, "text": "So here you go.", "start": 4988.534, "speaker": "SPEAKER_05"}, {"end": 4992.176, "text": "Just take it all.", "start": 4989.074, "speaker": "SPEAKER_05"}, {"end": 4997.939, "text": "This was very easy to do when we were designing desktop screens, because you just throw every big screen, throw everything you want in there.", "start": 4992.176, "speaker": "SPEAKER_05"}, {"end": 4998.26, "text": "I don't know.", "start": 4997.939, "speaker": "SPEAKER_05"}, {"end": 4999.26, "text": "What the hell do they want?", "start": 4998.26, "speaker": "SPEAKER_05"}, {"end": 4999.861, "text": "Horoscope?", "start": 4999.26, "speaker": "SPEAKER_05"}, {"end": 5000.821, "text": "No clue.", "start": 4999.861, "speaker": "SPEAKER_05"}, {"end": 5002.702, "text": "They probably want to see this woman dying.", "start": 5000.821, "speaker": "SPEAKER_05"}, {"end": 5003.143, "text": "I don't know.", "start": 5002.702, "speaker": "SPEAKER_05"}, {"end": 5005.704, "text": "Maybe how about the Netflix rating system?", "start": 5003.143, "speaker": "SPEAKER_05"}, {"end": 5006.064, "text": "Sure.", "start": 5005.704, "speaker": "SPEAKER_05"}, {"end": 5006.685, "text": "Throw that in there.", "start": 5006.064, "speaker": "SPEAKER_05"}, {"end": 5011.573, "text": "Let's throw it all in there and show everybody everything.", "start": 5008.09, "speaker": "SPEAKER_05"}, {"end": 5016.718, "text": "You can't do that on mobile because the screen is tiny.", "start": 5011.573, "speaker": "SPEAKER_05"}, {"end": 5021.043, "text": "And so you have to make these important things called decisions.", "start": 5016.718, "speaker": "SPEAKER_05"}, {"end": 5025.307, "text": "And mobile, in my mind, is all about getting people the right thing at the right time.", "start": 5021.043, "speaker": "SPEAKER_05"}, {"end": 5029.517, "text": "And so I'll just show you two simple examples of this.", "start": 5026.375, "speaker": "SPEAKER_05"}, {"end": 5031.257, "text": "Let's come back to messaging for a second.", "start": 5029.517, "speaker": "SPEAKER_05"}, {"end": 5034.679, "text": "So I get a note that says, what's your address?", "start": 5031.257, "speaker": "SPEAKER_05"}, {"end": 5041.822, "text": "And without me doing anything, there's two little buttons that show up right underneath the reply that say, work, home.", "start": 5034.679, "speaker": "SPEAKER_05"}, {"end": 5043.103, "text": "Please don't visit me.", "start": 5041.822, "speaker": "SPEAKER_05"}, {"end": 5047.065, "text": "And I just tap home.", "start": 5043.103, "speaker": "SPEAKER_05"}, {"end": 5049.166, "text": "And I'm done.", "start": 5047.065, "speaker": "SPEAKER_05"}, {"end": 5050.587, "text": "That's it.", "start": 5049.166, "speaker": "SPEAKER_05"}, {"end": 5053.368, "text": "This is a great example of the right thing at the right time.", "start": 5050.587, "speaker": "SPEAKER_05"}, {"end": 5062.021, "text": "And it's just like one tap, just like the Amazon Dash button, the Snapchat recording example.", "start": 5056.655, "speaker": "SPEAKER_05"}, {"end": 5066.165, "text": "This is why I think those natural user interfaces are good inspiration points.", "start": 5062.021, "speaker": "SPEAKER_05"}, {"end": 5068.167, "text": "They get us thinking like this.", "start": 5066.165, "speaker": "SPEAKER_05"}, {"end": 5079.053, "text": "They get us to think, well, what is the thing that right at that moment, at that right time, I think somebody echoed Jeff Bezos earlier, where he said, we start from the customer and iterate outward, right?", "start": 5068.167, "speaker": "SPEAKER_05"}, {"end": 5081.214, "text": "What is that customer perspective?", "start": 5079.053, "speaker": "SPEAKER_05"}, {"end": 5084.214, "text": "What is the need that people have, and how can we fulfill it?", "start": 5081.214, "speaker": "SPEAKER_05"}, {"end": 5088.935, "text": "Sticking with messaging, shifting over to Android, you get a message in a language you can't read?", "start": 5084.214, "speaker": "SPEAKER_05"}, {"end": 5089.835, "text": "No problem.", "start": 5088.935, "speaker": "SPEAKER_05"}, {"end": 5090.736, "text": "Just tap it.", "start": 5089.835, "speaker": "SPEAKER_05"}, {"end": 5092.716, "text": "You get tap to translate right there.", "start": 5090.736, "speaker": "SPEAKER_05"}, {"end": 5095.456, "text": "And without ever leaving the app, you can translate that text.", "start": 5092.716, "speaker": "SPEAKER_05"}, {"end": 5097.997, "text": "You can even then reply in that language.", "start": 5095.456, "speaker": "SPEAKER_05"}, {"end": 5106.193, "text": "It's another example of just a very frictionless, in the moment, right time, right action kind of UI.", "start": 5098.927, "speaker": "SPEAKER_05"}, {"end": 5109.876, "text": "And why do we care so much about these fast frictionless experiences?", "start": 5106.193, "speaker": "SPEAKER_05"}, {"end": 5113.799, "text": "Am I setting the bar too high?", "start": 5109.876, "speaker": "SPEAKER_05"}, {"end": 5115.921, "text": "Will this ever work for insurance?", "start": 5113.799, "speaker": "SPEAKER_05"}, {"end": 5118.363, "text": "Why do we care about such immediacy?", "start": 5115.921, "speaker": "SPEAKER_05"}, {"end": 5122.82, "text": "Many of you are probably thinking, well, yeah, OK, that's cool.", "start": 5120.777, "speaker": "SPEAKER_05"}, {"end": 5125.403, "text": "But things I do can't be like that.", "start": 5122.82, "speaker": "SPEAKER_05"}, {"end": 5126.704, "text": "My stuff is complicated.", "start": 5125.403, "speaker": "SPEAKER_05"}, {"end": 5128.907, "text": "It can't ever get simple.", "start": 5126.704, "speaker": "SPEAKER_05"}, {"end": 5133.933, "text": "And sure, maybe there are some things that can't or shouldn't necessarily get that simple.", "start": 5128.907, "speaker": "SPEAKER_05"}, {"end": 5137.557, "text": "But it sure can't hurt to try and go there, right?", "start": 5133.933, "speaker": "SPEAKER_05"}, {"end": 5142.024, "text": "Why do I raise the bar so high for mobile design?", "start": 5138.982, "speaker": "SPEAKER_05"}, {"end": 5147.467, "text": "Why do I care so much about these fast, frictionless, right time, right action interfaces?", "start": 5142.024, "speaker": "SPEAKER_05"}, {"end": 5151.309, "text": "Well, let's look at how people actually use their mobile devices.", "start": 5147.467, "speaker": "SPEAKER_05"}, {"end": 5153.971, "text": "Here we see mobile 3G traffic patterns.", "start": 5151.309, "speaker": "SPEAKER_05"}, {"end": 5157.473, "text": "So these are different kinds of devices connected to a cellular network.", "start": 5153.971, "speaker": "SPEAKER_05"}, {"end": 5160.473, "text": "And they're all on the same kind of network.", "start": 5158.792, "speaker": "SPEAKER_05"}, {"end": 5163.115, "text": "The only difference is there's different kinds of devices.", "start": 5160.473, "speaker": "SPEAKER_05"}, {"end": 5164.716, "text": "And so you can see laptops, tablets.", "start": 5163.115, "speaker": "SPEAKER_05"}, {"end": 5171.56, "text": "And what you notice about smartphones is there's a lot of these short, bursty sessions, a lot of in and out on the network.", "start": 5164.716, "speaker": "SPEAKER_05"}, {"end": 5177.387, "text": "contrast that to these bigger, longer, more sustained sessions on laptops.", "start": 5173.066, "speaker": "SPEAKER_05"}, {"end": 5182.389, "text": "And if you dig in a little bit further, Apple recently shared how often people log into their iPhone.", "start": 5177.387, "speaker": "SPEAKER_05"}, {"end": 5186.43, "text": "They said it's 80 times per day on average.", "start": 5182.389, "speaker": "SPEAKER_05"}, {"end": 5194.493, "text": "An on-device research study of Android found similar numbers, 76 times, and extreme users have higher numbers or so, but these are sort of averages.", "start": 5186.43, "speaker": "SPEAKER_05"}, {"end": 5195.553, "text": "So there's lots of sessions.", "start": 5194.493, "speaker": "SPEAKER_05"}, {"end": 5201.656, "text": "roughly like 80 something a day, which matches kind of what we see here, right?", "start": 5197.433, "speaker": "SPEAKER_05"}, {"end": 5204.978, "text": "Supports this data, lots of small singular sessions.", "start": 5201.656, "speaker": "SPEAKER_05"}, {"end": 5208.28, "text": "So lots of sessions happening on these devices.", "start": 5204.978, "speaker": "SPEAKER_05"}, {"end": 5213.663, "text": "And if you look at what these sessions look like, there are certainly some long ones.", "start": 5208.28, "speaker": "SPEAKER_05"}, {"end": 5214.724, "text": "That is very true.", "start": 5213.663, "speaker": "SPEAKER_05"}, {"end": 5218.446, "text": "Occasionally you watch Netflix, play Angry Birds or whatever.", "start": 5214.724, "speaker": "SPEAKER_05"}, {"end": 5221.428, "text": "But most fall, around 55% fall under 30 seconds or less.", "start": 5218.446, "speaker": "SPEAKER_05"}, {"end": 5227.043, "text": "And then the average is somewhere between like 80 to 120 seconds.", "start": 5223.893, "speaker": "SPEAKER_05"}, {"end": 5229.871, "text": "So it's not a lot of time per session.", "start": 5227.043, "speaker": "SPEAKER_05"}, {"end": 5231.436, "text": "Each session is really short.", "start": 5229.871, "speaker": "SPEAKER_05"}, {"end": 5237.376, "text": "There's a lot of them, but in general, they tend to be short.", "start": 5232.615, "speaker": "SPEAKER_05"}, {"end": 5243.838, "text": "And so understanding this context helps explain why I care so much about these fast, frictionless experiences.", "start": 5237.376, "speaker": "SPEAKER_05"}, {"end": 5247.92, "text": "I'm trying to design for the reality of how people use these devices.", "start": 5243.838, "speaker": "SPEAKER_05"}, {"end": 5252.181, "text": "They use them a lot, but they use them in small doses, quick chunks.", "start": 5247.92, "speaker": "SPEAKER_05"}, {"end": 5257.963, "text": "And so having this forced concentration on building for that reality helps a ton.", "start": 5252.181, "speaker": "SPEAKER_05"}, {"end": 5266.778, "text": "I think these numbers also illustrate, remember that talk about mobile performance, In this chart, what do people hate the most when browsing the web on mobile?", "start": 5257.963, "speaker": "SPEAKER_05"}, {"end": 5267.9, "text": "They hate slow things.", "start": 5266.778, "speaker": "SPEAKER_05"}, {"end": 5270.022, "text": "Why do they hate slow things?", "start": 5267.9, "speaker": "SPEAKER_05"}, {"end": 5272.325, "text": "Because they're only on the damn thing for 30 seconds.", "start": 5270.022, "speaker": "SPEAKER_05"}, {"end": 5279.395, "text": "If it takes 10 seconds to load, no wonder people dislike slow experiences.", "start": 5272.325, "speaker": "SPEAKER_05"}, {"end": 5288.324, "text": "And this is why solutions like things like skeleton screens, I figured I'd just throw this in here since we talked about performance earlier.", "start": 5280.839, "speaker": "SPEAKER_05"}, {"end": 5299.671, "text": "So skeleton screens are effectively these little baseline grids that make it look like content is coming in so you get a sense that things are actually happening, but they load very, very quickly because they don't grab images.", "start": 5288.324, "speaker": "SPEAKER_05"}, {"end": 5302.753, "text": "and actual content yet.", "start": 5300.531, "speaker": "SPEAKER_05"}, {"end": 5303.854, "text": "Actually, I'll show you how this works.", "start": 5302.753, "speaker": "SPEAKER_05"}, {"end": 5307.737, "text": "My favorite example of skeleton screens is from Google Photos.", "start": 5303.854, "speaker": "SPEAKER_05"}, {"end": 5311.639, "text": "So from Google Photos, when you load up a page, you get this grid.", "start": 5307.737, "speaker": "SPEAKER_05"}, {"end": 5314.321, "text": "It sort of shows you the relative proportion of images.", "start": 5311.639, "speaker": "SPEAKER_05"}, {"end": 5316.343, "text": "For example, you can see this is a panorama.", "start": 5314.321, "speaker": "SPEAKER_05"}, {"end": 5321.607, "text": "A little hard to see on the screen here, but it'll identify what's a video and how long it is.", "start": 5316.343, "speaker": "SPEAKER_05"}, {"end": 5325.43, "text": "So you get the structure of your content coming in instantly.", "start": 5321.607, "speaker": "SPEAKER_05"}, {"end": 5327.631, "text": "It's already there.", "start": 5326.33, "speaker": "SPEAKER_05"}, {"end": 5334.033, "text": "And then what we do is we gradually load in the content dynamically.", "start": 5327.631, "speaker": "SPEAKER_05"}, {"end": 5337.634, "text": "Well, we try to be smart, so we'll fade in things as they come in.", "start": 5334.033, "speaker": "SPEAKER_05"}, {"end": 5348.578, "text": "And then the other thing that happens is the content actually loads where you're scrolling to, not where you've been, so that it anticipates your movements and is there and ready for you when you show up.", "start": 5337.634, "speaker": "SPEAKER_05"}, {"end": 5351.086, "text": "So lots of things you can do.", "start": 5349.565, "speaker": "SPEAKER_05"}, {"end": 5353.947, "text": "And all of this stuff is around perception of speed.", "start": 5351.086, "speaker": "SPEAKER_05"}, {"end": 5358.868, "text": "There's a certain amount of speed that we can get from loading images and optimizing them and doing all that.", "start": 5353.947, "speaker": "SPEAKER_05"}, {"end": 5366.271, "text": "But then there's a lot of things that we can do in the user interface to make it feel like something's happening to create the perception of speed.", "start": 5358.868, "speaker": "SPEAKER_05"}, {"end": 5370.473, "text": "And again, another topic we can discuss in QA if you like.", "start": 5366.271, "speaker": "SPEAKER_05"}, {"end": 5370.573, "text": "OK.", "start": 5370.473, "speaker": "SPEAKER_05"}, {"end": 5376.571, "text": "Coming back to why these solutions exist, mobile interactions are frequent but short.", "start": 5372.329, "speaker": "SPEAKER_05"}, {"end": 5387.034, "text": "Another study that put cameras on people for a couple days and followed how long they looked at screens in their lives found it was about 38 seconds on average.", "start": 5376.571, "speaker": "SPEAKER_05"}, {"end": 5394.277, "text": "So similar data again, but the reason why I'm referencing the study is because what they were primarily looking at is smartwatches.", "start": 5387.034, "speaker": "SPEAKER_05"}, {"end": 5403.492, "text": "And they put cameras on people, they watched them throughout the day, and they saw how often they looked at their smartwatches and what they did on them.", "start": 5395.965, "speaker": "SPEAKER_05"}, {"end": 5407.476, "text": "And you'll find that the bar on these kinds of devices is even more extreme.", "start": 5403.492, "speaker": "SPEAKER_05"}, {"end": 5412.16, "text": "The average interaction time is about seven seconds.", "start": 5407.476, "speaker": "SPEAKER_05"}, {"end": 5415.103, "text": "Way more hardcore than mobile phones, right?", "start": 5412.16, "speaker": "SPEAKER_05"}, {"end": 5420.334, "text": "And I like this because it offers another opportunity to learn.", "start": 5416.089, "speaker": "SPEAKER_05"}, {"end": 5422.496, "text": "This is my continuing thing here.", "start": 5420.334, "speaker": "SPEAKER_05"}, {"end": 5428.042, "text": "Let's look at parallel stuff to see what can inspire us and can help us go even deeper on mobile.", "start": 5422.496, "speaker": "SPEAKER_05"}, {"end": 5433.888, "text": "Because if you are designing things for a 7-second world, you're going to kick some serious butt in the 30-second world.", "start": 5428.042, "speaker": "SPEAKER_05"}, {"end": 5438.912, "text": "So let's just look at a couple of these smartwatch interactions.", "start": 5435.57, "speaker": "SPEAKER_05"}, {"end": 5447.178, "text": "In this workout app, they're used to you run a workout, and then it tells you you can save or discard it, and then you have to either hit one of those buttons.", "start": 5438.912, "speaker": "SPEAKER_05"}, {"end": 5449.479, "text": "Sometimes you hit it on accident.", "start": 5447.178, "speaker": "SPEAKER_05"}, {"end": 5455.557, "text": "What happens in the Second version is it just automatically saves for you.", "start": 5449.479, "speaker": "SPEAKER_05"}, {"end": 5459.619, "text": "So a nice example of taking the burden on ourselves instead of putting it on customers.", "start": 5455.557, "speaker": "SPEAKER_05"}, {"end": 5461.901, "text": "And this is another way you can save a step.", "start": 5459.619, "speaker": "SPEAKER_05"}, {"end": 5462.942, "text": "You can save some effort.", "start": 5461.901, "speaker": "SPEAKER_05"}, {"end": 5463.943, "text": "You can save some thinking.", "start": 5462.942, "speaker": "SPEAKER_05"}, {"end": 5467.105, "text": "Just do things for people when they make sense.", "start": 5463.943, "speaker": "SPEAKER_05"}, {"end": 5469.447, "text": "You've been working out for 30 minutes or an hour?", "start": 5467.105, "speaker": "SPEAKER_05"}, {"end": 5472.709, "text": "I'm going to save this so you don't freak out when you lose it later.", "start": 5469.447, "speaker": "SPEAKER_05"}, {"end": 5475.951, "text": "You can discard it after the fact.", "start": 5472.709, "speaker": "SPEAKER_05"}, {"end": 5483.346, "text": "Sticking with the same workout app for half a second, The first time you use it, you get this series of choices.", "start": 5475.951, "speaker": "SPEAKER_05"}, {"end": 5485.668, "text": "You can either do a walk, run, cycle.", "start": 5483.346, "speaker": "SPEAKER_05"}, {"end": 5496.456, "text": "Second time you use it, what it'll do is learn what you did the first time, and it'll put the last type of activity that you did at the front and make it nice and big.", "start": 5485.668, "speaker": "SPEAKER_05"}, {"end": 5499.799, "text": "So this is an example of learning from past behavior.", "start": 5496.456, "speaker": "SPEAKER_05"}, {"end": 5501.22, "text": "You've done something once.", "start": 5499.799, "speaker": "SPEAKER_05"}, {"end": 5502.841, "text": "We don't need you to repeat it yet again.", "start": 5501.22, "speaker": "SPEAKER_05"}, {"end": 5503.722, "text": "Here you go.", "start": 5502.841, "speaker": "SPEAKER_05"}, {"end": 5508.224, "text": "Shifting over to another app, this is a little timer app.", "start": 5505.442, "speaker": "SPEAKER_05"}, {"end": 5515.83, "text": "And in the previous design, you had to meticulously pick how long you want the timer to run, and then you hit start, and then it finally starts.", "start": 5508.224, "speaker": "SPEAKER_05"}, {"end": 5520.774, "text": "In the next redesign, what they did is they just made these super fast and simple defaults.", "start": 5515.83, "speaker": "SPEAKER_05"}, {"end": 5526.298, "text": "Customization is down lower if you need it, but you just tap it and you're up and running.", "start": 5520.774, "speaker": "SPEAKER_05"}, {"end": 5530.722, "text": "So this is a nice example of setting some custom frictionless defaults.", "start": 5526.298, "speaker": "SPEAKER_05"}, {"end": 5534.472, "text": "And this last one I think is, or is this the last one?", "start": 5532.091, "speaker": "SPEAKER_05"}, {"end": 5535.112, "text": "Maybe the second one.", "start": 5534.472, "speaker": "SPEAKER_05"}, {"end": 5539.734, "text": "This one's my favorite because this actually changed how I use this device.", "start": 5535.112, "speaker": "SPEAKER_05"}, {"end": 5547.578, "text": "So if you looked at the way they did messaging before, you had to tap to get the option to reply to someone's message.", "start": 5539.734, "speaker": "SPEAKER_05"}, {"end": 5549.018, "text": "And then you hit reply.", "start": 5547.578, "speaker": "SPEAKER_05"}, {"end": 5552.38, "text": "And then you get a choice of how to reply.", "start": 5549.018, "speaker": "SPEAKER_05"}, {"end": 5557.702, "text": "In the new version, right there below the message, you get the little microphone.", "start": 5552.38, "speaker": "SPEAKER_05"}, {"end": 5565.706, "text": "Super fast and easy to actually respond.", "start": 5560.883, "speaker": "SPEAKER_05"}, {"end": 5569.349, "text": "Before, when you had to tap multiple times, it was like, oh, I might as well just get my phone.", "start": 5565.706, "speaker": "SPEAKER_05"}, {"end": 5571.03, "text": "Now, you just go, boom.", "start": 5569.349, "speaker": "SPEAKER_05"}, {"end": 5574.252, "text": "And instantly, it's faster and easier than actually using your phone.", "start": 5571.03, "speaker": "SPEAKER_05"}, {"end": 5580.487, "text": "Again, small stuff, small changes, but it's shifted kind of the behavior.", "start": 5575.544, "speaker": "SPEAKER_05"}, {"end": 5581.088, "text": "And I lied.", "start": 5580.487, "speaker": "SPEAKER_05"}, {"end": 5583.61, "text": "One more smartwatch example for a second.", "start": 5581.088, "speaker": "SPEAKER_05"}, {"end": 5594.257, "text": "Again, in this case for inspiration, this is one of the things that I think illustrate to me how natural user interfaces and software and all the capabilities of these devices can come together.", "start": 5583.61, "speaker": "SPEAKER_05"}, {"end": 5597.239, "text": "Because right now, I can get a light little tap on my wrist.", "start": 5594.257, "speaker": "SPEAKER_05"}, {"end": 5599.26, "text": "There's a haptic engine that sort of taps me.", "start": 5597.239, "speaker": "SPEAKER_05"}, {"end": 5599.621, "text": "It goes, bup.", "start": 5599.26, "speaker": "SPEAKER_05"}, {"end": 5601.863, "text": "tells me something's new.", "start": 5600.441, "speaker": "SPEAKER_05"}, {"end": 5609.17, "text": "I twist my wrist up, it uses the accelerometer on the device to know it's changed position and tell me, here's what I blipped you about.", "start": 5601.863, "speaker": "SPEAKER_05"}, {"end": 5612.194, "text": "This satellite is about to pass overhead and you can see it.", "start": 5609.17, "speaker": "SPEAKER_05"}, {"end": 5613.855, "text": "Since you're an astronomy geek, cool.", "start": 5612.194, "speaker": "SPEAKER_05"}, {"end": 5615.427, "text": "Right?", "start": 5614.826, "speaker": "SPEAKER_05"}, {"end": 5623.174, "text": "So I just tap on it once, and it uses the magnimeter, the digital compass, to show me where that satellite is relative to me.", "start": 5615.427, "speaker": "SPEAKER_05"}, {"end": 5628.819, "text": "And if I want to, I can tap in again, and now it uses GPS to say, here's where you are.", "start": 5623.174, "speaker": "SPEAKER_05"}, {"end": 5629.94, "text": "There's that street.", "start": 5628.819, "speaker": "SPEAKER_05"}, {"end": 5633.283, "text": "You should look in that direction.", "start": 5629.94, "speaker": "SPEAKER_05"}, {"end": 5634.344, "text": "That's pretty freaking cool.", "start": 5633.283, "speaker": "SPEAKER_05"}, {"end": 5643.297, "text": "It's using all this stuff, the GPS, the magnimeter, the accelerometer, the haptic engine, and all this data and technology behind it.", "start": 5635.573, "speaker": "SPEAKER_05"}, {"end": 5646.259, "text": "But to me, it just feels like, hey, satellite, where?", "start": 5643.297, "speaker": "SPEAKER_05"}, {"end": 5649.14, "text": "Oh, right there.", "start": 5646.259, "speaker": "SPEAKER_05"}, {"end": 5651.402, "text": "If you like to look at satellites.", "start": 5649.14, "speaker": "SPEAKER_05"}, {"end": 5652.482, "text": "Some of you might not.", "start": 5651.402, "speaker": "SPEAKER_05"}, {"end": 5658.045, "text": "My kids get a thrill of seeing the ISS fly overhead all the time, so it's pretty neat.", "start": 5652.482, "speaker": "SPEAKER_05"}, {"end": 5664.843, "text": "Anyway, what can we learn from smartwatch app design, where this time to interaction is super extreme?", "start": 5659.241, "speaker": "SPEAKER_05"}, {"end": 5670.905, "text": "We looked at ways to minimize effort, learn from past behavior, and use these sensors, all that good stuff.", "start": 5664.843, "speaker": "SPEAKER_05"}, {"end": 5674.186, "text": "And all of these are really good insights that we can then bring to mobile.", "start": 5670.905, "speaker": "SPEAKER_05"}, {"end": 5681.409, "text": "And so I'm just going to end with two examples of doing that, and then leave you to break, and we can talk about all this good stuff going forward.", "start": 5674.186, "speaker": "SPEAKER_05"}, {"end": 5684.129, "text": "So things like pressure-sensitive inputs.", "start": 5681.409, "speaker": "SPEAKER_05"}, {"end": 5693.072, "text": "If I'm looking at a list of music, and I do this thing where I just push a little harder, Right then and there, I can go add it to a playlist, download it, delete it, remove it.", "start": 5684.129, "speaker": "SPEAKER_05"}, {"end": 5697.613, "text": "It's this really fast and easy shortcut inspired by a quick little gesture.", "start": 5693.072, "speaker": "SPEAKER_05"}, {"end": 5706.296, "text": "Of course, I can still tap regularly, go to the page about the song, tap the little meatball menu, and then get those choices.", "start": 5697.613, "speaker": "SPEAKER_05"}, {"end": 5711.557, "text": "It's certainly there because the other gesture is a little hidden, but this nice, fast, and fluid shortcut helps a lot.", "start": 5706.296, "speaker": "SPEAKER_05"}, {"end": 5716.104, "text": "Here's another example of these sort of fast and fluid shortcuts.", "start": 5713.321, "speaker": "SPEAKER_05"}, {"end": 5720.367, "text": "And this is a technique called fluid gestures, fluid touch gestures.", "start": 5716.104, "speaker": "SPEAKER_05"}, {"end": 5723.25, "text": "So here what I do is I just press and hold on the microphone.", "start": 5720.367, "speaker": "SPEAKER_05"}, {"end": 5724.691, "text": "It starts recording.", "start": 5723.25, "speaker": "SPEAKER_05"}, {"end": 5726.573, "text": "I just move up and let go.", "start": 5724.691, "speaker": "SPEAKER_05"}, {"end": 5729.155, "text": "Audio gets sent.", "start": 5726.573, "speaker": "SPEAKER_05"}, {"end": 5730.876, "text": "So again, I just press and hold.", "start": 5729.155, "speaker": "SPEAKER_05"}, {"end": 5732.338, "text": "Starts recording as I talk.", "start": 5730.876, "speaker": "SPEAKER_05"}, {"end": 5733.219, "text": "I move my finger up.", "start": 5732.338, "speaker": "SPEAKER_05"}, {"end": 5735.04, "text": "gets sent.", "start": 5733.94, "speaker": "SPEAKER_05"}, {"end": 5739.342, "text": "One fast, fluid touch gesture that does multiple things at once.", "start": 5735.04, "speaker": "SPEAKER_05"}, {"end": 5744.144, "text": "And contrast this type of fluid gesture to something like the current UI in Maps.", "start": 5739.342, "speaker": "SPEAKER_05"}, {"end": 5746.105, "text": "You have to touch Search.", "start": 5744.144, "speaker": "SPEAKER_05"}, {"end": 5748.225, "text": "You go and say, I want to go home.", "start": 5746.105, "speaker": "SPEAKER_05"}, {"end": 5749.206, "text": "And then it comes back.", "start": 5748.225, "speaker": "SPEAKER_05"}, {"end": 5750.506, "text": "And you can say Start.", "start": 5749.206, "speaker": "SPEAKER_05"}, {"end": 5756.068, "text": "It's this multi-tap, very GUI-like experience for getting things done.", "start": 5750.506, "speaker": "SPEAKER_05"}, {"end": 5760.81, "text": "And if you contrast this to a fluid touch experience, you just hold down Search.", "start": 5756.068, "speaker": "SPEAKER_05"}, {"end": 5762.07, "text": "Up pops a list of options.", "start": 5760.81, "speaker": "SPEAKER_05"}, {"end": 5763.091, "text": "Drag up.", "start": 5762.07, "speaker": "SPEAKER_05"}, {"end": 5763.391, "text": "Let go.", "start": 5763.091, "speaker": "SPEAKER_05"}, {"end": 5766.783, "text": "and you're navigating away.", "start": 5764.42, "speaker": "SPEAKER_05"}, {"end": 5778.595, "text": "Again, more inspiration than anything else, but when you start thinking about these fast, fluid, frictionless interactions, you can see how you can start thinking about making things that work like that.", "start": 5766.783, "speaker": "SPEAKER_05"}, {"end": 5793.851, "text": "All that aside, hopefully some of these examples from hardware, from smart watches, from natural user interfaces, from fluid touch gestures, one of those things I hope has inspired some of you, because that's the point here.", "start": 5781.095, "speaker": "SPEAKER_05"}, {"end": 5799.958, "text": "And the point is we need to know the experience we want in the future so that we can work toward it today.", "start": 5793.851, "speaker": "SPEAKER_05"}, {"end": 5806.527, "text": "Mobile has created this huge opportunity for us, for audience growth, for revenue.", "start": 5801.501, "speaker": "SPEAKER_05"}, {"end": 5810.411, "text": "All these devices out there are at global scale.", "start": 5806.527, "speaker": "SPEAKER_05"}, {"end": 5814.075, "text": "Huge area open to us for deployment.", "start": 5810.411, "speaker": "SPEAKER_05"}, {"end": 5820.368, "text": "And we can, using a bunch of the techniques I showed you today, optimize for today.", "start": 5815.926, "speaker": "SPEAKER_05"}, {"end": 5826.831, "text": "There's ways to take critical interactions like checkout, critical interactions like sign-in, and make them a lot better.", "start": 5820.368, "speaker": "SPEAKER_05"}, {"end": 5829.712, "text": "You can go and make these improvements right now.", "start": 5826.831, "speaker": "SPEAKER_05"}, {"end": 5837.895, "text": "And you can probably end up in a state like Amazon or Target that I showed, Walmart and Target that I showed earlier, where they're double the industry average, right?", "start": 5829.712, "speaker": "SPEAKER_05"}, {"end": 5841.295, "text": "Instead of 30%, they're at 60% sales on mobile.", "start": 5837.895, "speaker": "SPEAKER_05"}, {"end": 5845.116, "text": "All this stuff is possible today, provided you put in the effort.", "start": 5841.295, "speaker": "SPEAKER_05"}, {"end": 5850.137, "text": "But we also need to look past today, toward the future.", "start": 5845.116, "speaker": "SPEAKER_05"}, {"end": 5861.519, "text": "And if we look to the future, I think there's an opportunity to remove this friction even further, to take all this technical stuff and put it in the service of human beings, so we can just go live our lives.", "start": 5850.137, "speaker": "SPEAKER_05"}, {"end": 5863.62, "text": "So we can just do what comes naturally to us.", "start": 5861.519, "speaker": "SPEAKER_05"}, {"end": 5868.724, "text": "And we use our understanding of technology to enable that.", "start": 5864.58, "speaker": "SPEAKER_05"}, {"end": 5880.074, "text": "And if you have a vision like that, I think it helps with the optimization and iteration process, because you kind of know where you're trying to get, and you can evaluate your steps.", "start": 5868.724, "speaker": "SPEAKER_05"}, {"end": 5884.438, "text": "The future, I think, is going to be even more seamless if we do our jobs right.", "start": 5880.074, "speaker": "SPEAKER_05"}, {"end": 5890.243, "text": "The other point I'll make is the next 10 years, high degree of likelihood are going to move even faster than the previous 10.", "start": 5884.438, "speaker": "SPEAKER_05"}, {"end": 5899.785, "text": "So the sooner we get ready for them, the sooner we start thinking about them, the sooner we start aiming for them, the better off we will be when they arrive.", "start": 5891.499, "speaker": "SPEAKER_05"}, {"end": 5903.668, "text": "And those are kind of the three big things I want to leave you with.", "start": 5899.785, "speaker": "SPEAKER_05"}, {"end": 5908.852, "text": "So thanks.", "start": 5903.668, "speaker": "SPEAKER_05"}, {"end": 5919.761, "text": "So as promised, first part was me blabbing on for a while.", "start": 5916.137, "speaker": "SPEAKER_05"}, {"end": 5922.263, "text": "Second part is us talking.", "start": 5919.761, "speaker": "SPEAKER_05"}, {"end": 5927.648, "text": "So we have this microphone, which I will do my best to throw properly.", "start": 5922.263, "speaker": "SPEAKER_05"}, {"end": 5930.511, "text": "And I am here to answer your questions.", "start": 5927.648, "speaker": "SPEAKER_05"}, {"end": 5936.157, "text": "Hopefully, there's things that are on your mind that we can talk through that will help you.", "start": 5930.511, "speaker": "SPEAKER_05"}, {"end": 5937.438, "text": "And who's up first?", "start": 5936.157, "speaker": "SPEAKER_05"}, {"end": 5944.155, "text": "Either I can't see, or you're all way too shy.", "start": 5941.334, "speaker": "SPEAKER_05"}, {"end": 5945.215, "text": "OK.", "start": 5944.155, "speaker": "SPEAKER_05"}, {"end": 5947.135, "text": "All right, here we go.", "start": 5945.215, "speaker": "SPEAKER_05"}, {"end": 5949.176, "text": "Ready for it?", "start": 5947.135, "speaker": "SPEAKER_05"}, {"end": 5952.336, "text": "Oh, that's not even close.", "start": 5949.176, "speaker": "SPEAKER_05"}, {"end": 5956.057, "text": "You have to ask a question.", "start": 5952.336, "speaker": "SPEAKER_05"}, {"end": 5956.977, "text": "Double-handed.", "start": 5956.057, "speaker": "SPEAKER_05"}, {"end": 5958.398, "text": "Didn't even work.", "start": 5956.977, "speaker": "SPEAKER_05"}, {"end": 5959.338, "text": "All right.", "start": 5958.398, "speaker": "SPEAKER_11"}, {"end": 5960.318, "text": "OK.", "start": 5959.338, "speaker": "SPEAKER_05"}, {"end": 5963.899, "text": "Well, you were talking a lot about the user interfaces.", "start": 5960.318, "speaker": "SPEAKER_11"}, {"end": 5966.96, "text": "What about the gesture-based user interfaces?", "start": 5963.899, "speaker": "SPEAKER_11"}, {"end": 5968.26, "text": "Any nice examples of those?", "start": 5966.96, "speaker": "SPEAKER_11"}, {"end": 5971.734, "text": "Yeah, so what about gesture-based stuff?", "start": 5969.232, "speaker": "SPEAKER_05"}, {"end": 5983.383, "text": "I talked a little bit about some of those things, and I think there's a number of things to consider when we talk about gesture-based interfaces, so let me find some stuff.", "start": 5971.734, "speaker": "SPEAKER_05"}, {"end": 5987.692411764706, "text": "So one thing that I talked about... ", "start": 5983.383, "speaker": "SPEAKER_05"}, {"end": 5989.488, "text": "Oh, here we go.", "start": 5987.692411764706, "speaker": "SPEAKER_05"}, {"end": 5999.801, "text": "One thing that I talked about is understanding how people actually use devices and making sure our designs work with that.", "start": 5992.117, "speaker": "SPEAKER_05"}, {"end": 6004.163, "text": "So just a couple of fun facts, stats, whatever that can help inform you.", "start": 5999.801, "speaker": "SPEAKER_05"}, {"end": 6013.888, "text": "Number one, if you look at smartphone browsing behavior globally, you'll find that most of it happens in this vertical orientation.", "start": 6004.163, "speaker": "SPEAKER_05"}, {"end": 6018.434, "text": "And this is for people browsing web pages and things like this.", "start": 6014.932, "speaker": "SPEAKER_05"}, {"end": 6027.679, "text": "So what we take from this is trying to think about what is the ergonomic consideration of a gesture, right?", "start": 6018.434, "speaker": "SPEAKER_05"}, {"end": 6031.361, "text": "What position would make gestures actually work and are comfortable?", "start": 6027.679, "speaker": "SPEAKER_05"}, {"end": 6035.305, "text": "And I also talked a little bit about one-handed use.", "start": 6031.881, "speaker": "SPEAKER_05"}, {"end": 6040.432, "text": "If you go and take a look at how people use devices, there's a couple common patterns.", "start": 6035.305, "speaker": "SPEAKER_05"}, {"end": 6041.793, "text": "There's the one-handed use.", "start": 6040.432, "speaker": "SPEAKER_05"}, {"end": 6044.477, "text": "There's the, I need help for my one hand with my other hand.", "start": 6041.793, "speaker": "SPEAKER_05"}, {"end": 6047.48, "text": "And then there's our friend, the Crackberry Prayer, right?", "start": 6044.477, "speaker": "SPEAKER_05"}, {"end": 6054.593, "text": "analysis of people doing this.", "start": 6051.112, "speaker": "SPEAKER_05"}, {"end": 6058.835, "text": "This comes from 1,100 records of people out in the wild.", "start": 6054.593, "speaker": "SPEAKER_05"}, {"end": 6061.696, "text": "We see half the time, it's one-handed like this.", "start": 6058.835, "speaker": "SPEAKER_05"}, {"end": 6064.897, "text": "About a third of the time, it's like this.", "start": 6061.696, "speaker": "SPEAKER_05"}, {"end": 6068.378, "text": "And much less so are they doing the crackberry prayer.", "start": 6064.897, "speaker": "SPEAKER_05"}, {"end": 6074.601, "text": "Now, if you drill down a little deeper, what you actually find is this two-handed use comes in two flavors.", "start": 6068.378, "speaker": "SPEAKER_05"}, {"end": 6076.041, "text": "One is one finger like this.", "start": 6074.601, "speaker": "SPEAKER_05"}, {"end": 6078.062, "text": "The other one is a thumb like that.", "start": 6076.041, "speaker": "SPEAKER_05"}, {"end": 6085.178, "text": "And this breakdown, almost three-fourths of the time, they're like that, and 28% of the time, they're like that.", "start": 6079.395, "speaker": "SPEAKER_05"}, {"end": 6092.302, "text": "So cumulatively, what we see is 75% were really thinking about one hand and a thumb.", "start": 6085.178, "speaker": "SPEAKER_05"}, {"end": 6098.665, "text": "And I think that forms a lot of the basis for how to think about which gestures are comfortable and easy to use.", "start": 6092.302, "speaker": "SPEAKER_05"}, {"end": 6102.487, "text": "To illustrate that, I have a video of me stalking a woman on an airplane.", "start": 6098.665, "speaker": "SPEAKER_05"}, {"end": 6107.006, "text": "So here I am filming her without her knowing.", "start": 6104.505, "speaker": "SPEAKER_05"}, {"end": 6108.206, "text": "I'd probably get arrested for this.", "start": 6107.006, "speaker": "SPEAKER_05"}, {"end": 6111.327, "text": "But you can see how she's just using the thumb here.", "start": 6108.206, "speaker": "SPEAKER_05"}, {"end": 6113.028, "text": "And this clearly looks like Facebook.", "start": 6111.327, "speaker": "SPEAKER_05"}, {"end": 6114.668, "text": "So she's going to flick, flick, flick.", "start": 6113.028, "speaker": "SPEAKER_05"}, {"end": 6116.229, "text": "And then she tries to tap, tap.", "start": 6114.668, "speaker": "SPEAKER_05"}, {"end": 6117.309, "text": "She does the two-hand thing.", "start": 6116.229, "speaker": "SPEAKER_05"}, {"end": 6123.672, "text": "Tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap.", "start": 6117.309, "speaker": "SPEAKER_05"}, {"end": 6124.532, "text": "Screw it.", "start": 6123.672, "speaker": "SPEAKER_05"}, {"end": 6128.893, "text": "It's back to flicking.", "start": 6124.532, "speaker": "SPEAKER_05"}, {"end": 6141.363, "text": "I like that video because it illustrates, A, that I'm creepy, but B, It shows the behavior I was talking about there in the abstract form very realistically.", "start": 6128.893, "speaker": "SPEAKER_05"}, {"end": 6142.864, "text": "It's that one hand, one thumb.", "start": 6141.363, "speaker": "SPEAKER_05"}, {"end": 6148.089, "text": "You saw her put her additional hand up there so she could get to the higher edges of the screen.", "start": 6142.864, "speaker": "SPEAKER_05"}, {"end": 6156.797, "text": "And then the other thing you noticed is touch target sizes, pretty broken in that experience because they're probably just small link-based things in tiny text.", "start": 6148.089, "speaker": "SPEAKER_05"}, {"end": 6159.238, "text": "and very difficult to hit.", "start": 6157.377, "speaker": "SPEAKER_05"}, {"end": 6166.42, "text": "I think Craig talked about the scientific chart around gestures, and I have some stuff on that somewhere I could probably dig up.", "start": 6159.238, "speaker": "SPEAKER_05"}, {"end": 6175.183, "text": "But in general, you want to make sure you have the right touch target sizes for hands and design for that accordingly.", "start": 6166.42, "speaker": "SPEAKER_05"}, {"end": 6180.924, "text": "Now, what does it mean when we design for these kinds of use cases?", "start": 6175.183, "speaker": "SPEAKER_05"}, {"end": 6184.245, "text": "Let me see if I've got a nice image.", "start": 6180.924, "speaker": "SPEAKER_05"}, {"end": 6186.326, "text": "Probably do, but let's see here.", "start": 6184.245, "speaker": "SPEAKER_05"}, {"end": 6191.128, "text": "And for reference, what I'm doing is scanning like 600 slides.", "start": 6188.005, "speaker": "SPEAKER_05"}, {"end": 6195.432, "text": "So sometimes it takes me a second to find the appropriate thing.", "start": 6191.128, "speaker": "SPEAKER_05"}, {"end": 6204.76, "text": "Let's open up another deck real quick and find it there.", "start": 6195.432, "speaker": "SPEAKER_05"}, {"end": 6210.965, "text": "One of these will be it.", "start": 6204.76, "speaker": "SPEAKER_05"}, {"end": 6211.425, "text": "Let's see.", "start": 6210.965, "speaker": "SPEAKER_05"}, {"end": 6216.463, "text": "Not that one.", "start": 6215.863, "speaker": "SPEAKER_05"}, {"end": 6227.007, "text": "I got a nice video of making use of those types of gestures and illustrating how they can work for you.", "start": 6216.463, "speaker": "SPEAKER_05"}, {"end": 6231.168, "text": "So give me a secundo.", "start": 6227.007, "speaker": "SPEAKER_05"}, {"end": 6235.43, "text": "See, I should have some jokes prepared for situations like this, right?", "start": 6231.168, "speaker": "SPEAKER_05"}, {"end": 6239.871, "text": "Where I just sit there scrolling, scrolling, scrolling, awkwardly.", "start": 6235.43, "speaker": "SPEAKER_05"}, {"end": 6241.552, "text": "And you say, what the hell is he doing?", "start": 6239.871, "speaker": "SPEAKER_05"}, {"end": 6242.712, "text": "And I say, I don't know.", "start": 6241.552, "speaker": "SPEAKER_05"}, {"end": 6248.628, "text": "All right, one more deck, and then I will just jump to this other piece.", "start": 6244.466, "speaker": "SPEAKER_05"}, {"end": 6250.269, "text": "Screen, screen, screen.", "start": 6248.628, "speaker": "SPEAKER_05"}, {"end": 6266.978, "text": "All right, if this isn't it, I give up, and I'll show you some other stuff.", "start": 6250.269, "speaker": "SPEAKER_05"}, {"end": 6267.898, "text": "OK, I give up.", "start": 6266.978, "speaker": "SPEAKER_05"}, {"end": 6268.799, "text": "I'll show you some other stuff.", "start": 6267.898, "speaker": "SPEAKER_05"}, {"end": 6273.423, "text": "Actually, we can maybe talk about this.", "start": 6271.782, "speaker": "SPEAKER_05"}, {"end": 6274.444, "text": "This is somewhat related.", "start": 6273.423, "speaker": "SPEAKER_05"}, {"end": 6279.327, "text": "Let me show you this and then I'll switch over to the other thing.", "start": 6274.444, "speaker": "SPEAKER_05"}, {"end": 6282.228, "text": "Some examples of how we can account for this kind of behavior.", "start": 6279.327, "speaker": "SPEAKER_05"}, {"end": 6297.328, "text": "For example, here, this principle that I've talked about for a while, and excuse the dated example of the old style here, but there's A general principle that, and I think this got mentioned a few times yesterday, how typing is hard on mobile.", "start": 6282.228, "speaker": "SPEAKER_05"}, {"end": 6299.569, "text": "I think Ani talked about this a bunch of times.", "start": 6297.328, "speaker": "SPEAKER_05"}, {"end": 6305.35, "text": "There's a number of things that you can do to make typing easier and also take advantage of gestures for it.", "start": 6299.569, "speaker": "SPEAKER_05"}, {"end": 6315.194, "text": "So here when you're creating this guide, there's default text up there, there's how to, and then there's these little buttony experiences where you can just tap install.", "start": 6305.35, "speaker": "SPEAKER_05"}, {"end": 6318.155, "text": "So how to install, and you're off to the races.", "start": 6315.194, "speaker": "SPEAKER_05"}, {"end": 6323.944, "text": "So we generally have this principle of try not to let the keyboard come up.", "start": 6318.955, "speaker": "SPEAKER_05"}, {"end": 6327.531, "text": "Is there other ways that people can do things without a keyboard?", "start": 6323.944, "speaker": "SPEAKER_05"}, {"end": 6328.332, "text": "And Google Trips.", "start": 6327.531, "speaker": "SPEAKER_05"}, {"end": 6331.414, "text": "has some nice examples for this.", "start": 6329.392, "speaker": "SPEAKER_05"}, {"end": 6339.461, "text": "So when you are searching for a flight destination, for example, we're going to default the airport closest to you using geolocation.", "start": 6331.414, "speaker": "SPEAKER_05"}, {"end": 6342.223, "text": "So you don't have to enter the airport closest to you.", "start": 6339.461, "speaker": "SPEAKER_05"}, {"end": 6347.667, "text": "We're going to give you suggestions of what are the most popular destinations from where you're at.", "start": 6342.223, "speaker": "SPEAKER_05"}, {"end": 6350.61, "text": "So from San Jose, people usually go to Las Vegas.", "start": 6347.667, "speaker": "SPEAKER_05"}, {"end": 6353.693, "text": "So again, you can just pick one of those things by default.", "start": 6350.61, "speaker": "SPEAKER_05"}, {"end": 6355.934, "text": "And you don't have to type in your to destination.", "start": 6353.693, "speaker": "SPEAKER_05"}, {"end": 6369.001, "text": "If you want to pick an airport here, again, rather than getting you to type, you can just hit that little location icon.", "start": 6356.875, "speaker": "SPEAKER_05"}, {"end": 6370.622, "text": "It'll give you other airports near you.", "start": 6369.001, "speaker": "SPEAKER_05"}, {"end": 6376.125, "text": "So if you want to change from San Jose to, say, San Francisco, you can do that without the keyboard coming up.", "start": 6370.622, "speaker": "SPEAKER_05"}, {"end": 6378.418, "text": "So there you go.", "start": 6377.097, "speaker": "SPEAKER_05"}, {"end": 6382.603, "text": "Now, if you want to enter a destination, you can tap that map icon.", "start": 6378.418, "speaker": "SPEAKER_05"}, {"end": 6387.848, "text": "And again, you can scroll all the way around here and find a destination.", "start": 6382.603, "speaker": "SPEAKER_05"}, {"end": 6389.83, "text": "These are kind of the most common places things go.", "start": 6387.848, "speaker": "SPEAKER_05"}, {"end": 6391.611, "text": "And you can just tap like that.", "start": 6389.83, "speaker": "SPEAKER_05"}, {"end": 6396.516, "text": "And it fills in that information, starts to tell you how much it's going to cost.", "start": 6391.611, "speaker": "SPEAKER_05"}, {"end": 6397.257, "text": "And you're right back.", "start": 6396.516, "speaker": "SPEAKER_05"}, {"end": 6400.519, "text": "And you're right back here.", "start": 6398.515, "speaker": "SPEAKER_05"}, {"end": 6404.225, "text": "And now what we've done, you can see there's already results coming in.", "start": 6400.519, "speaker": "SPEAKER_05"}, {"end": 6409.334, "text": "So if one of these suits your fancy, you're ready to go because it's sort of defaulted into a...", "start": 6404.225, "speaker": "SPEAKER_05"}, {"end": 6414.07, "text": "March 8th through 12th thing, but you can tap on that date.", "start": 6411.028, "speaker": "SPEAKER_05"}, {"end": 6419.994, "text": "And when you're in the date, here there's a nice big scrolling area, which again, you can kind of use with one thumb.", "start": 6414.07, "speaker": "SPEAKER_05"}, {"end": 6426.659, "text": "So you can tap on 8 to 12, or however you want, and contrast that experience.", "start": 6419.994, "speaker": "SPEAKER_05"}, {"end": 6428.24, "text": "So we'll show it in video form here.", "start": 6426.659, "speaker": "SPEAKER_05"}, {"end": 6432.463, "text": "So I tap on date, and I can just scroll through the months really quickly and easily.", "start": 6428.24, "speaker": "SPEAKER_05"}, {"end": 6437.286, "text": "I can tap the 3rd, and then I can tap the 13th or 13th to the 1st.", "start": 6432.463, "speaker": "SPEAKER_05"}, {"end": 6438.547, "text": "Tells me how much it costs.", "start": 6437.286, "speaker": "SPEAKER_05"}, {"end": 6445.45, "text": "I tap done and I've already picked my dates through this nice fluid one thumb kind of UI.", "start": 6439.267, "speaker": "SPEAKER_05"}, {"end": 6453.753, "text": "Contrast this to something where somebody ports over like a desktop pattern and this is kind of your standard desktop widget where you flip between the months, right?", "start": 6445.45, "speaker": "SPEAKER_05"}, {"end": 6463.337, "text": "And you can pick March 20th and then you tap again to get your return date and now you move through the months this way and you can go back here and tap a date there.", "start": 6453.753, "speaker": "SPEAKER_05"}, {"end": 6467.962, "text": "In this case, we'll tap February 7th or February 20th.", "start": 6464.76, "speaker": "SPEAKER_05"}, {"end": 6474.647, "text": "And now we're departing March 20th and returning February 21st.", "start": 6467.962, "speaker": "SPEAKER_05"}, {"end": 6475.748, "text": "Anybody see a problem with that?", "start": 6474.647, "speaker": "SPEAKER_05"}, {"end": 6480.466, "text": "But so this previous UI did a couple things.", "start": 6477.824, "speaker": "SPEAKER_05"}, {"end": 6483.848, "text": "A, it took advantage of this one thumb kind of behavior, right?", "start": 6480.466, "speaker": "SPEAKER_05"}, {"end": 6486.35, "text": "Scroll up and down, tap ones, tap the other.", "start": 6483.848, "speaker": "SPEAKER_05"}, {"end": 6489.532, "text": "But it also prevented errors from happening here.", "start": 6486.35, "speaker": "SPEAKER_05"}, {"end": 6498.899, "text": "And so this is, I think, thinking more for this kind of UI as opposed to porting over your desktop UI, which I feel this is an example of.", "start": 6489.532, "speaker": "SPEAKER_05"}, {"end": 6509.623, "text": "And hopefully this example illustrates kind of the contrast of, let me just take my website, shrink it to a small screen versus let me think about the reality of gestures.", "start": 6499.659, "speaker": "SPEAKER_05"}, {"end": 6511.664, "text": "And there's other things that happen here.", "start": 6509.623, "speaker": "SPEAKER_05"}, {"end": 6519.887, "text": "If you want to change price or time duration, you get a nice slider, which also is sort of optimized for those kinds of interactions.", "start": 6511.664, "speaker": "SPEAKER_05"}, {"end": 6521.548, "text": "So you can flick it over here.", "start": 6519.887, "speaker": "SPEAKER_05"}, {"end": 6533.493, "text": "And when you add all this stuff up, right, that's about seven different things that are being done in this small form UI to try to keep you off the keyboard, try to prevent you from typing.", "start": 6522.724, "speaker": "SPEAKER_05"}, {"end": 6561.192, "text": "Use location, use the map, use that date picker, use sliders, all these kinds of interactions that are trying to help build for the reality that I illustrated earlier, which, again, looks something like And this is one of the things that I think we should always keep in mind, which is what is the reality of use?", "start": 6533.493, "speaker": "SPEAKER_05"}, {"end": 6563.394, "text": "What are people actually doing?", "start": 6561.192, "speaker": "SPEAKER_05"}, {"end": 6567.256, "text": "Similar to those short bursty interactions, the number of times they interact.", "start": 6563.394, "speaker": "SPEAKER_05"}, {"end": 6569.538, "text": "How are they actually holding the phone?", "start": 6567.256, "speaker": "SPEAKER_05"}, {"end": 6571.339, "text": "How are thumbs sized?", "start": 6569.538, "speaker": "SPEAKER_05"}, {"end": 6572.22, "text": "What can they do?", "start": 6571.339, "speaker": "SPEAKER_05"}, {"end": 6573, "text": "What can't they do?", "start": 6572.22, "speaker": "SPEAKER_05"}, {"end": 6579.045, "text": "All that can inform what we do with design and gestures.", "start": 6573, "speaker": "SPEAKER_05"}, {"end": 6579.385, "text": "Sound good?", "start": 6579.045, "speaker": "SPEAKER_05"}, {"end": 6582.811, "text": "OK, he gave me the thumbs up.", "start": 6581.128, "speaker": "SPEAKER_05"}, {"end": 6585.375, "text": "All right, where else should we throw the microphone?", "start": 6582.811, "speaker": "SPEAKER_05"}, {"end": 6586.577, "text": "I saw there's one behind you.", "start": 6585.375, "speaker": "SPEAKER_05"}, {"end": 6587.319, "text": "Yep, two behind.", "start": 6586.577, "speaker": "SPEAKER_05"}, {"end": 6603.095, "text": "Yeah, could you talk about using drop-down inputs and not using drop-down inputs?", "start": 6595.249, "speaker": "SPEAKER_09"}, {"end": 6606.518, "text": "Can we talk about drop-downs and not drop-downs?", "start": 6603.095, "speaker": "SPEAKER_05"}, {"end": 6614.034892045454, "text": "Especially on stuff that's not like when you can't use location, but a set of... ", "start": 6606.518, "speaker": "SPEAKER_09"}, {"end": 6622.851, "text": "We have to pick one of five options or one of 50 options and alternatives to using a drop-down.", "start": 6614.034892045454, "speaker": "SPEAKER_09"}, {"end": 6623.892, "text": "What happened?", "start": 6622.851, "speaker": "SPEAKER_05"}, {"end": 6624.032, "text": "Ta-da!", "start": 6623.892, "speaker": "SPEAKER_05"}, {"end": 6629.254, "text": "OK, let's talk about dropdowns.", "start": 6627.933, "speaker": "SPEAKER_05"}, {"end": 6633.337, "text": "Just so happens I have 50 slides on dropdowns.", "start": 6629.254, "speaker": "SPEAKER_05"}, {"end": 6634.418, "text": "I don't know how that happened.", "start": 6633.337, "speaker": "SPEAKER_05"}, {"end": 6644.848, "text": "I have this, so let me back up half a second.", "start": 6642.027, "speaker": "SPEAKER_05"}, {"end": 6651.089, "text": "A lot of the ways I think about doing design is under this notion of a consideration, right?", "start": 6644.848, "speaker": "SPEAKER_05"}, {"end": 6656.93, "text": "As opposed to hard and fast rules, like this is the way you have to do this, or this is the answer.", "start": 6651.089, "speaker": "SPEAKER_05"}, {"end": 6663.751, "text": "Instead, I like to present things as considerations, which is stuff to think about as you're going through the design process.", "start": 6656.93, "speaker": "SPEAKER_05"}, {"end": 6670.332, "text": "While you're building a mobile experience, what kinds of ideas or aspirations should you have?", "start": 6663.751, "speaker": "SPEAKER_05"}, {"end": 6678.916, "text": "And one of these design considerations, aka something to think about while designing, is treating dropdowns as a last resort.", "start": 6671.112, "speaker": "SPEAKER_05"}, {"end": 6681.857, "text": "And why do I say this?", "start": 6678.916, "speaker": "SPEAKER_05"}, {"end": 6684.739, "text": "So let's look at, in the US, picking a state.", "start": 6681.857, "speaker": "SPEAKER_05"}, {"end": 6687.38, "text": "So you tap the state thing.", "start": 6684.739, "speaker": "SPEAKER_05"}, {"end": 6691.042, "text": "And if you live in Alabama, this is pretty much the only good thing that happens to you.", "start": 6687.38, "speaker": "SPEAKER_05"}, {"end": 6699.61, "text": "Because you get to use these drop downs of just two taps.", "start": 6695.889, "speaker": "SPEAKER_05"}, {"end": 6705.673, "text": "For everybody else, they have to scroll and find their state, and then that could be multiple taps, and then they have to tap done.", "start": 6699.61, "speaker": "SPEAKER_05"}, {"end": 6708.434, "text": "So it's at least four taps in most cases.", "start": 6705.673, "speaker": "SPEAKER_05"}, {"end": 6709.054, "text": "Kind of painful.", "start": 6708.434, "speaker": "SPEAKER_05"}, {"end": 6714.976, "text": "And remember this example I showed earlier, sometimes drop-downs get used to an extreme.", "start": 6709.935, "speaker": "SPEAKER_05"}, {"end": 6719.118, "text": "So in this form, departure date is three drop-downs.", "start": 6714.976, "speaker": "SPEAKER_05"}, {"end": 6721.878, "text": "Return date is three drop-downs.", "start": 6719.118, "speaker": "SPEAKER_05"}, {"end": 6724.119, "text": "Number of passengers is a drop-down.", "start": 6721.878, "speaker": "SPEAKER_05"}, {"end": 6726, "text": "Cabin preference is a drop-down.", "start": 6724.119, "speaker": "SPEAKER_05"}, {"end": 6727.56, "text": "Country of residence is a drop-down.", "start": 6726, "speaker": "SPEAKER_05"}, {"end": 6729.741, "text": "Drop-down, drop-down, drop-down, drop-down.", "start": 6727.56, "speaker": "SPEAKER_05"}, {"end": 6734.062, "text": "And many people's design processes, this is the thing they reach for first.", "start": 6729.741, "speaker": "SPEAKER_05"}, {"end": 6737.743, "text": "It's like their hammer, or their nail, or whatever.", "start": 6734.802, "speaker": "SPEAKER_05"}, {"end": 6739.424, "text": "They always reach for the dropdown.", "start": 6737.743, "speaker": "SPEAKER_05"}, {"end": 6745.407, "text": "But dropdowns, in many cases, are quite hard to navigate, as we saw earlier, especially on desktops.", "start": 6739.424, "speaker": "SPEAKER_05"}, {"end": 6751.65, "text": "If you've ever gotten into a long dropdown, try precisely picking that thing out of the list.", "start": 6745.407, "speaker": "SPEAKER_05"}, {"end": 6753.791, "text": "You can't really do any editing.", "start": 6751.65, "speaker": "SPEAKER_05"}, {"end": 6755.812, "text": "It doesn't support hierarchy, and on and on.", "start": 6753.791, "speaker": "SPEAKER_05"}, {"end": 6763.965, "text": "So I showed this pattern earlier where you type a zip code and you get back a city and state.", "start": 6757.736, "speaker": "SPEAKER_05"}, {"end": 6766.789, "text": "In some cases, you may get back more than one.", "start": 6763.965, "speaker": "SPEAKER_05"}, {"end": 6770.475, "text": "But instead of presenting them as a dropdown, we can present them as a little stepper.", "start": 6766.789, "speaker": "SPEAKER_05"}, {"end": 6776.85, "text": "And so a stepper would allow you to move through those options quickly and easily with a nice little touch gesture.", "start": 6771.385, "speaker": "SPEAKER_05"}, {"end": 6780.534, "text": "And so a stepper is one alternative to a dropdown.", "start": 6776.85, "speaker": "SPEAKER_05"}, {"end": 6783.276, "text": "And what a stepper will do is move things up or down.", "start": 6780.534, "speaker": "SPEAKER_05"}, {"end": 6786.699, "text": "It'll sort of change a value through a consistent set of adjustments.", "start": 6783.276, "speaker": "SPEAKER_05"}, {"end": 6792.985, "text": "And you can adjust the stepper to actually have an input value inside if you want.", "start": 6786.699, "speaker": "SPEAKER_05"}, {"end": 6798.263, "text": "And there's lots of opportunities where these kinds of interfaces make a ton of sense.", "start": 6792.985, "speaker": "SPEAKER_05"}, {"end": 6805.647, "text": "So here, when you're picking a number of travelers in a booking form, you can just go, yeah, two.", "start": 6798.263, "speaker": "SPEAKER_05"}, {"end": 6807.688, "text": "The most travelers you can have is eight.", "start": 6805.647, "speaker": "SPEAKER_05"}, {"end": 6813.711, "text": "So in worst case scenario, you'll tap this thing seven times.", "start": 6807.688, "speaker": "SPEAKER_05"}, {"end": 6814.671, "text": "Here's another example.", "start": 6813.711, "speaker": "SPEAKER_05"}, {"end": 6825.597, "text": "Instead of using a dropdown, there's this multi-picker menu for the number of children, number of infants, which maybe it's easier just to do something like this.", "start": 6814.671, "speaker": "SPEAKER_05"}, {"end": 6834.742, "text": "And if you look at Airbnb, they used to have guests and beds and bedrooms and bathrooms all done with dropdown menus.", "start": 6827.517, "speaker": "SPEAKER_05"}, {"end": 6841.406, "text": "And now they've moved over to these little stepper controls here inside of their Android app.", "start": 6834.742, "speaker": "SPEAKER_05"}, {"end": 6844.048, "text": "Now, there's good and bad ways to use steppers.", "start": 6841.406, "speaker": "SPEAKER_05"}, {"end": 6847.09, "text": "Just there is with every single UI element.", "start": 6844.048, "speaker": "SPEAKER_05"}, {"end": 6851.373, "text": "So here on American Airlines, in their redesigned form, they do use a stepper.", "start": 6847.09, "speaker": "SPEAKER_05"}, {"end": 6864.713, "text": "if you look at this thing, but I think there's a couple issues with this one, which is A, these things don't really look actionable because they're so light gray and off to the sides, right?", "start": 6853.867, "speaker": "SPEAKER_05"}, {"end": 6867.394, "text": "It doesn't really feel like an actionable UI.", "start": 6864.713, "speaker": "SPEAKER_05"}, {"end": 6869.695, "text": "And it's so separated from this.", "start": 6867.394, "speaker": "SPEAKER_05"}, {"end": 6879.22, "text": "And then if you go into this mode, if you will, let me just hit that, you enter like stepper mania.", "start": 6869.695, "speaker": "SPEAKER_05"}, {"end": 6890.112, "text": "And again, if these things felt a little more like actual stepper controls, I think it would be better.", "start": 6883.688, "speaker": "SPEAKER_05"}, {"end": 6896.235, "text": "But they kind of feel inactive, and it's unclear what's actually happening here.", "start": 6890.112, "speaker": "SPEAKER_05"}, {"end": 6903.199, "text": "Now, another place that we saw dropdowns being used was sort of airport class of travel.", "start": 6896.235, "speaker": "SPEAKER_05"}, {"end": 6905.76, "text": "So in this case, economy of restrictions.", "start": 6903.199, "speaker": "SPEAKER_05"}, {"end": 6911.223, "text": "And if you tap on that, then you can see all of your options, including business class and higher.", "start": 6905.76, "speaker": "SPEAKER_05"}, {"end": 6912.043, "text": "Business class and higher.", "start": 6911.223, "speaker": "SPEAKER_05"}, {"end": 6916.993, "text": "First, premium infrastructure.", "start": 6915.312, "speaker": "SPEAKER_05"}, {"end": 6924.057, "text": "By the way, do you think there's lawyers that work at American Airlines?", "start": 6916.993, "speaker": "SPEAKER_05"}, {"end": 6927.019, "text": "I'm pretty sure a lawyer wrote that copy.", "start": 6924.057, "speaker": "SPEAKER_05"}, {"end": 6937.476, "text": "Contrast that to what happens on British Airways, where they actually use this thing called English to describe There are travel classes, economy, premium economy, business.", "start": 6927.019, "speaker": "SPEAKER_05"}, {"end": 6939.158, "text": "But this is a little action sheet.", "start": 6937.476, "speaker": "SPEAKER_05"}, {"end": 6943.522, "text": "So this pops up from the bottom, and you can pick the type of class of travel you have.", "start": 6939.158, "speaker": "SPEAKER_05"}, {"end": 6945.604, "text": "And an action sheet is another alternative here.", "start": 6943.522, "speaker": "SPEAKER_05"}, {"end": 6949.408, "text": "This is kind of that bottom menu thing I showed you earlier.", "start": 6945.604, "speaker": "SPEAKER_05"}, {"end": 6953.752, "text": "So you have a set of choices related to some task.", "start": 6949.408, "speaker": "SPEAKER_05"}, {"end": 6954.873, "text": "It comes up from the bottom.", "start": 6953.752, "speaker": "SPEAKER_05"}, {"end": 6956.995, "text": "You can have a cancel action.", "start": 6954.873, "speaker": "SPEAKER_05"}, {"end": 6957.736, "text": "Works real nice.", "start": 6956.995, "speaker": "SPEAKER_05"}, {"end": 6972.495, "text": "Looking at United, the way they ask class of travel is you tap on that and you come into this full page view.", "start": 6963.253, "speaker": "SPEAKER_05"}, {"end": 6977.076, "text": "Which, though they're using English, this seems kind of excessive, right?", "start": 6972.495, "speaker": "SPEAKER_05"}, {"end": 6986.099, "text": "But United, it turns out, is super into lists because even picking the number of passengers results in a list.", "start": 6977.076, "speaker": "SPEAKER_05"}, {"end": 6987.479, "text": "So they love them some lists.", "start": 6986.099, "speaker": "SPEAKER_05"}, {"end": 6995.182, "text": "Contrast the way all those folks ask for a class of travel to the way this site does it.", "start": 6988.819, "speaker": "SPEAKER_05"}, {"end": 6998.124, "text": "I think this is Hipmunk, if I remember correctly.", "start": 6995.182, "speaker": "SPEAKER_05"}, {"end": 7007.488, "text": "And here they've reduced a class of travel down to a series of segmented controls, or in the Android world, radio button groups.", "start": 6998.124, "speaker": "SPEAKER_05"}, {"end": 7012.27, "text": "So a radio button group is a set of related but mutually exclusive choices.", "start": 7007.488, "speaker": "SPEAKER_05"}, {"end": 7014.652, "text": "You generally have five or less of them.", "start": 7012.27, "speaker": "SPEAKER_05"}, {"end": 7017.113, "text": "And it's also a nice contender for this kind of stuff.", "start": 7014.652, "speaker": "SPEAKER_05"}, {"end": 7022.201, "text": "Now, we looked at dates earlier, so I'll just sort of jump through dates really quickly.", "start": 7017.797, "speaker": "SPEAKER_05"}, {"end": 7030.067, "text": "But in general, these kinds of desktop date pickers don't work nearly as well as something that's optimized more for mobile.", "start": 7022.201, "speaker": "SPEAKER_05"}, {"end": 7033.71, "text": "And you're starting to see more people adapt to these kinds of controls.", "start": 7030.067, "speaker": "SPEAKER_05"}, {"end": 7037.052, "text": "Although, I don't know why Kayak makes it look like a real calendar.", "start": 7033.71, "speaker": "SPEAKER_05"}, {"end": 7041.375, "text": "This looks like a printed calendar here.", "start": 7037.052, "speaker": "SPEAKER_05"}, {"end": 7044.378, "text": "And I don't know why United inserts this big gap here.", "start": 7041.375, "speaker": "SPEAKER_05"}, {"end": 7051.49, "text": "But increasingly, people are moving more towards single calendar things, which we'll see in a second.", "start": 7045.607, "speaker": "SPEAKER_05"}, {"end": 7057.574, "text": "And once you start thinking about all the different places and ways you can replace dropdowns, there's a whole world of opportunity.", "start": 7051.49, "speaker": "SPEAKER_05"}, {"end": 7058.694, "text": "You can use switches.", "start": 7057.574, "speaker": "SPEAKER_05"}, {"end": 7059.855, "text": "You can use sliders.", "start": 7058.694, "speaker": "SPEAKER_05"}, {"end": 7062.537, "text": "You can use segmented controls.", "start": 7059.855, "speaker": "SPEAKER_05"}, {"end": 7067.54, "text": "Switches will let you do on-off type things, mutually exclusive choices.", "start": 7062.537, "speaker": "SPEAKER_05"}, {"end": 7069.941, "text": "Sliders will let you move through a range of stuff.", "start": 7067.54, "speaker": "SPEAKER_05"}, {"end": 7076.653, "text": "There's good and bad ways to use each of these controls.", "start": 7072.89, "speaker": "SPEAKER_05"}, {"end": 7084.178, "text": "For example, I wouldn't necessarily use a slider to pick the number of people in a hotel room.", "start": 7076.653, "speaker": "SPEAKER_05"}, {"end": 7086.7, "text": "And you can style sliders however you want.", "start": 7084.178, "speaker": "SPEAKER_05"}, {"end": 7089.322, "text": "So there's a whole slew of things available to you, right?", "start": 7086.7, "speaker": "SPEAKER_05"}, {"end": 7094.489, "text": "steppers, switches, segmented controls, sliders, date pickers, dialogues.", "start": 7090.226, "speaker": "SPEAKER_05"}, {"end": 7099.433, "text": "A lot of these controls are built for touch interfaces.", "start": 7094.489, "speaker": "SPEAKER_05"}, {"end": 7109.34, "text": "And they're almost, you know, they're available in some ways on desktops, but a lot of them have been built with touch target sizes in mind.", "start": 7099.433, "speaker": "SPEAKER_05"}, {"end": 7115.324, "text": "And coming back to, let me just jump a couple slides to get to the point.", "start": 7109.34, "speaker": "SPEAKER_05"}, {"end": 7118.126, "text": "Let me do it a faster way.", "start": 7115.324, "speaker": "SPEAKER_05"}, {"end": 7118.627, "text": "Where are we?", "start": 7118.126, "speaker": "SPEAKER_05"}, {"end": 7121.104, "text": "Here we are.", "start": 7120.683, "speaker": "SPEAKER_05"}, {"end": 7125.688, "text": "At least I thought I was over there.", "start": 7121.104, "speaker": "SPEAKER_05"}, {"end": 7128.911, "text": "So let's look at these things in action.", "start": 7125.688, "speaker": "SPEAKER_05"}, {"end": 7130.493, "text": "So what's the point?", "start": 7128.911, "speaker": "SPEAKER_05"}, {"end": 7134.117, "text": "Here's a bunch of different ways of doing what we did with dropdowns.", "start": 7130.493, "speaker": "SPEAKER_05"}, {"end": 7138.801, "text": "We wanted to see, are these actually better than dropdowns in many cases?", "start": 7134.117, "speaker": "SPEAKER_05"}, {"end": 7150.106, "text": "And so what we did is we took a bunch of apps for booking flight travel, and we specifically picked examples that use different controls to ask different questions.", "start": 7139.522, "speaker": "SPEAKER_05"}, {"end": 7154.327, "text": "And we compared which controls worked best for each question.", "start": 7150.106, "speaker": "SPEAKER_05"}, {"end": 7163.111, "text": "Because what it turns out is these are your list of options, but different types of questions involve different types of answers.", "start": 7154.327, "speaker": "SPEAKER_05"}, {"end": 7166.292, "text": "So different controls are better fit for different types of questions.", "start": 7163.111, "speaker": "SPEAKER_05"}, {"end": 7176.57, "text": "To illustrate this, it turns out choosing travel dates on a single calendar was easier and quicker than having these two calendar things.", "start": 7167.6, "speaker": "SPEAKER_05"}, {"end": 7181.676, "text": "So this pick-your-travel-dates UI that I showed you worked really well.", "start": 7176.57, "speaker": "SPEAKER_05"}, {"end": 7187.023, "text": "For picking number of passengers, the most efficient and easy-to-use thing turned out to be the stepper.", "start": 7181.676, "speaker": "SPEAKER_05"}, {"end": 7190.488, "text": "This resonated really well with people.", "start": 7188.147, "speaker": "SPEAKER_05"}, {"end": 7195.651, "text": "When it came to picking class of travel, the action sheet worked quite good.", "start": 7190.488, "speaker": "SPEAKER_05"}, {"end": 7197.912, "text": "When you tap on it, up comes your options.", "start": 7195.651, "speaker": "SPEAKER_05"}, {"end": 7200.994, "text": "But then also, the segmented control worked really well, too.", "start": 7197.912, "speaker": "SPEAKER_05"}, {"end": 7204.115, "text": "It was almost like a tie there.", "start": 7200.994, "speaker": "SPEAKER_05"}, {"end": 7207.617, "text": "And here, people kind of appreciated seeing all of the options front and center.", "start": 7204.115, "speaker": "SPEAKER_05"}, {"end": 7212.721, "text": "And so, cumulatively, all these things, I think, really add up.", "start": 7209.018, "speaker": "SPEAKER_05"}, {"end": 7219.706, "text": "If you remember the image I showed earlier, when you've got this drop-down written form, your first instinct is this is gonna suck.", "start": 7212.721, "speaker": "SPEAKER_05"}, {"end": 7232.234, "text": "Whereas if you have a form that uses the appropriate types of inputs for the kinds of questions you're asking, then you can really make something that feels like it'll be a lot easier to do, right?", "start": 7219.706, "speaker": "SPEAKER_05"}, {"end": 7233.335, "text": "A lot more approachable.", "start": 7232.234, "speaker": "SPEAKER_05"}, {"end": 7234.176, "text": "This is pretty easy.", "start": 7233.335, "speaker": "SPEAKER_05"}, {"end": 7235.196, "text": "All my stuff is right here.", "start": 7234.176, "speaker": "SPEAKER_05"}, {"end": 7237.418, "text": "I can jump in here and interact with it.", "start": 7235.196, "speaker": "SPEAKER_05"}, {"end": 7247.593, "text": "Now, the question I always get asked about drop-down apps, about drop-downs, is, well, that's OK for easy stuff, right?", "start": 7238.387, "speaker": "SPEAKER_05"}, {"end": 7255.038, "text": "It's easy to find a flight that puts you in a metal tin can that'll transport you anywhere in the world in hours.", "start": 7247.593, "speaker": "SPEAKER_05"}, {"end": 7260.262, "text": "But what about complicated things, like enterprise-y stuff?", "start": 7255.038, "speaker": "SPEAKER_05"}, {"end": 7261.863, "text": "I think the same rules apply.", "start": 7260.262, "speaker": "SPEAKER_05"}, {"end": 7267.547, "text": "In fact, this list of what makes drop-down menus hard to use I got from a company building enterprise apps.", "start": 7261.863, "speaker": "SPEAKER_05"}, {"end": 7277.686, "text": "And what they illustrated is how they moved to a new series of controls to help people manage their invoicing system.", "start": 7268.795, "speaker": "SPEAKER_05"}, {"end": 7280.93, "text": "And so they installed this system called the Picker.", "start": 7277.686, "speaker": "SPEAKER_05"}, {"end": 7284.995, "text": "And this runs across their mobile, tablet, and desktop UIs.", "start": 7280.93, "speaker": "SPEAKER_05"}, {"end": 7292.612, "text": "And if you hit the invoice due date, What you actually get is a little picker control that allows you to pick when things are due.", "start": 7284.995, "speaker": "SPEAKER_05"}, {"end": 7296.116, "text": "You get a bunch of default options, so you can do this quickly and easily.", "start": 7292.612, "speaker": "SPEAKER_05"}, {"end": 7301.901, "text": "Or you can go and set a specific date, which, again, is quite hard to do with a dropdown menu.", "start": 7296.116, "speaker": "SPEAKER_05"}, {"end": 7307.506, "text": "You can pick a specific date, and then it'll come back and fill that in.", "start": 7301.901, "speaker": "SPEAKER_05"}, {"end": 7312.691, "text": "And because they built a responsive design, this works in kind of a layered fashion on small screens as well.", "start": 7307.506, "speaker": "SPEAKER_05"}, {"end": 7322.791, "text": "Looking at another one of the elements inside of this form, if you want to change the units in the invoice, you don't want to have pieces.", "start": 7314.864, "speaker": "SPEAKER_05"}, {"end": 7325.313, "text": "You want to have something else.", "start": 7322.791, "speaker": "SPEAKER_05"}, {"end": 7330.518, "text": "Not only can you pick between the defaults, but there is a search field at the top.", "start": 7325.313, "speaker": "SPEAKER_05"}, {"end": 7332.319, "text": "So you can go in and type kilowatt.", "start": 7330.518, "speaker": "SPEAKER_05"}, {"end": 7335.342, "text": "You can type or tap kilowatt per hour.", "start": 7332.319, "speaker": "SPEAKER_05"}, {"end": 7335.882, "text": "There it is.", "start": 7335.342, "speaker": "SPEAKER_05"}, {"end": 7342.388, "text": "And because they have these custom controls, next time you tap it, kilowatt hour sits at the top and the list gets updated.", "start": 7335.882, "speaker": "SPEAKER_05"}, {"end": 7351.134, "text": "So it's an example of how alternatives to dropdowns can be applied to even things that are, quote unquote, more complex.", "start": 7343.95, "speaker": "SPEAKER_05"}, {"end": 7367.003, "text": "And the reason why I harp on this stuff is, again, all these other mobile optimized controls can really help reduce input effort, make things feel more approachable, and basically get to the right question and answer pairing in terms of UI.", "start": 7351.134, "speaker": "SPEAKER_05"}, {"end": 7374.448, "text": "And yes, there are situations where you might want to use a dropdown, but I prefer to treat it as the last resort.", "start": 7367.763, "speaker": "SPEAKER_05"}, {"end": 7376.089, "text": "I can't do it with a stepper.", "start": 7374.448, "speaker": "SPEAKER_05"}, {"end": 7377.711, "text": "I can't do it with this.", "start": 7376.089, "speaker": "SPEAKER_05"}, {"end": 7382.534, "text": "Let me come back to the dropdown as my last choice.", "start": 7377.711, "speaker": "SPEAKER_05"}, {"end": 7387.057, "text": "OK, I got another thumbs up.", "start": 7382.534, "speaker": "SPEAKER_05"}, {"end": 7387.278, "text": "Yay.", "start": 7387.057, "speaker": "SPEAKER_05"}, {"end": 7390.76, "text": "All right, where should the microphone go next?", "start": 7387.278, "speaker": "SPEAKER_05"}, {"end": 7394.243, "text": "OK, it looks like in the middle here.", "start": 7390.76, "speaker": "SPEAKER_05"}, {"end": 7394.703, "text": "Over there?", "start": 7394.243, "speaker": "SPEAKER_05"}, {"end": 7395.304, "text": "Yeah?", "start": 7394.703, "speaker": "SPEAKER_05"}, {"end": 7395.784, "text": "OK, cool.", "start": 7395.304, "speaker": "SPEAKER_05"}, {"end": 7397.125, "text": "We'll make our way down gradually.", "start": 7395.784, "speaker": "SPEAKER_05"}, {"end": 7403.027, "text": "Luke, question on contextual design.", "start": 7399.144, "speaker": "SPEAKER_01"}, {"end": 7408.45, "text": "Google's been working very hard on answering questions on the first page of the search results.", "start": 7403.027, "speaker": "SPEAKER_01"}, {"end": 7411.652, "text": "How would you design something that I haven't asked yet?", "start": 7408.45, "speaker": "SPEAKER_01"}, {"end": 7414.314, "text": "How would I design something that you haven't asked yet?", "start": 7411.652, "speaker": "SPEAKER_05"}, {"end": 7416.976, "text": "I'm standing at the bus stop getting the phone out of my pocket.", "start": 7414.314, "speaker": "SPEAKER_01"}, {"end": 7420.298, "text": "I want to see what time it takes until the bus arrives.", "start": 7416.976, "speaker": "SPEAKER_01"}, {"end": 7421.298, "text": "Have you been working on that?", "start": 7420.298, "speaker": "SPEAKER_01"}, {"end": 7424.06, "text": "Yeah, how will I know what you're going to ask?", "start": 7421.298, "speaker": "SPEAKER_05"}, {"end": 7426.602, "text": "I have this mind reading device implanted in my ear.", "start": 7424.06, "speaker": "SPEAKER_05"}, {"end": 7430.91, "text": "I think there's a bunch of signals that we can use.", "start": 7428.108, "speaker": "SPEAKER_05"}, {"end": 7434.233, "text": "Context, though, is a very challenging thing to infer.", "start": 7430.91, "speaker": "SPEAKER_05"}, {"end": 7442.339, "text": "And so the question is, can we kind of infer what people think or what they're trying to do and maybe have preemptive interfaces?", "start": 7434.233, "speaker": "SPEAKER_05"}, {"end": 7445.141, "text": "It's a very nice aspirational pattern.", "start": 7442.339, "speaker": "SPEAKER_05"}, {"end": 7447.243, "text": "And I do have something somewhere on it.", "start": 7445.141, "speaker": "SPEAKER_05"}, {"end": 7448.143, "text": "Yeah, I do.", "start": 7447.243, "speaker": "SPEAKER_05"}, {"end": 7452.787, "text": "Let me bring this up and show you.", "start": 7448.143, "speaker": "SPEAKER_05"}, {"end": 7454.328, "text": "It's inside this other deck.", "start": 7452.787, "speaker": "SPEAKER_05"}, {"end": 7455.449, "text": "So give me half a second.", "start": 7454.328, "speaker": "SPEAKER_05"}, {"end": 7462.934, "text": "The problem with inferring things is many times we are still wrong.", "start": 7457.912, "speaker": "SPEAKER_05"}, {"end": 7466.376, "text": "We just don't have enough data to know necessarily what's happening.", "start": 7462.934, "speaker": "SPEAKER_05"}, {"end": 7468.757, "text": "And you can see an example of this.", "start": 7466.376, "speaker": "SPEAKER_05"}, {"end": 7481.062, "text": "If you go back to early days of mobile, did anybody ever end up on a mobile site that thought it knew you were a mobile user and only showed you the things that it thought were good for mobile users?", "start": 7468.757, "speaker": "SPEAKER_05"}, {"end": 7481.542, "text": "Remember this?", "start": 7481.062, "speaker": "SPEAKER_05"}, {"end": 7486.465, "text": "And I was like, here's the limited dumb dumb thing for you, poor mobile user.", "start": 7482.602, "speaker": "SPEAKER_05"}, {"end": 7491.348, "text": "If only you had a large screen device, I could tell you everything about myself.", "start": 7486.465, "speaker": "SPEAKER_05"}, {"end": 7494.73, "text": "And so it kind of inferred that, oh, someone's on mobile.", "start": 7491.348, "speaker": "SPEAKER_05"}, {"end": 7496.051, "text": "They're probably on the go.", "start": 7494.73, "speaker": "SPEAKER_05"}, {"end": 7499.394, "text": "So we're only going to show them this and this info.", "start": 7496.051, "speaker": "SPEAKER_05"}, {"end": 7503.536, "text": "The reality is people will use their mobile devices everywhere and anywhere.", "start": 7499.394, "speaker": "SPEAKER_05"}, {"end": 7504.637, "text": "You're all using it now.", "start": 7503.536, "speaker": "SPEAKER_05"}, {"end": 7506.378, "text": "Are you mobile?", "start": 7504.637, "speaker": "SPEAKER_05"}, {"end": 7508.8, "text": "Or are you just farting around on Facebook?", "start": 7506.378, "speaker": "SPEAKER_05"}, {"end": 7514.124, "text": "I saw a couple of guilty as charged faces.", "start": 7511.803, "speaker": "SPEAKER_05"}, {"end": 7521.609, "text": "And so even though we know you're stationary and indoors and in an office, we don't necessarily know 100% your intent.", "start": 7514.124, "speaker": "SPEAKER_05"}, {"end": 7523.91, "text": "So let me show you a way of dealing with that.", "start": 7521.609, "speaker": "SPEAKER_05"}, {"end": 7529.133, "text": "And I think I have a nice series of images, if I can get to them.", "start": 7523.91, "speaker": "SPEAKER_05"}, {"end": 7531.354, "text": "There we go.", "start": 7529.133, "speaker": "SPEAKER_05"}, {"end": 7538.744, "text": "Now, this is the way I used to always illustrate what I was talking about before when this was an issue.", "start": 7533.743, "speaker": "SPEAKER_05"}, {"end": 7549.247, "text": "Luckily, this isn't so much of an issue, because it used to be when we thought about taking our websites down to mobile, the question we had was, well, what do we get rid of from our website so we can make mobile websites?", "start": 7538.744, "speaker": "SPEAKER_05"}, {"end": 7552.347, "text": "What sort of limited content and features will we get?", "start": 7549.247, "speaker": "SPEAKER_05"}, {"end": 7561.905, "text": "And what you ended up with stuff is having Marriott, which has this big website about all of their different products and their destinations.", "start": 7552.347, "speaker": "SPEAKER_05"}, {"end": 7569.511, "text": "And then you had mobile version of Marriott, which was this simplified dumbed down kind of thing.", "start": 7561.905, "speaker": "SPEAKER_05"}, {"end": 7573.013, "text": "And when you tried to get here, so here I'm going to tap getting here.", "start": 7569.511, "speaker": "SPEAKER_05"}, {"end": 7579.538, "text": "Mm-hmm.", "start": 7573.013, "speaker": "SPEAKER_09"}, {"end": 7593.844, "text": "Or as it turns out, on the desktop version, they had this thing called Airport Shuttle Service that would actually tell you how to get to the hotel.", "start": 7584.307, "speaker": "SPEAKER_05"}, {"end": 7595.847, "text": "But that wasn't on mobile.", "start": 7593.844, "speaker": "SPEAKER_05"}, {"end": 7602.438, "text": "because they had cut that part out and what have you, even though it's actually technically useful information for mobile.", "start": 7596.673, "speaker": "SPEAKER_05"}, {"end": 7606.021, "text": "So that's kind of an illustration of what used to happen.", "start": 7602.438, "speaker": "SPEAKER_05"}, {"end": 7608.983, "text": "And hopefully, that doesn't really happen that much anymore.", "start": 7606.021, "speaker": "SPEAKER_05"}, {"end": 7616.75, "text": "I haven't had to talk about that in a long time, which leads me to think maybe it's OK.", "start": 7608.983, "speaker": "SPEAKER_05"}, {"end": 7619.432, "text": "But let me show you the other piece of the coin here.", "start": 7616.75, "speaker": "SPEAKER_05"}, {"end": 7622.114, "text": "And I think I have a nice example.", "start": 7619.432, "speaker": "SPEAKER_05"}, {"end": 7622.995, "text": "If I can find it.", "start": 7622.114, "speaker": "SPEAKER_05"}, {"end": 7632.281, "text": "Almost there.", "start": 7630.94, "speaker": "SPEAKER_05"}, {"end": 7633.581, "text": "Yes.", "start": 7632.281, "speaker": "SPEAKER_05"}, {"end": 7635.062, "text": "Here we go.", "start": 7633.581, "speaker": "SPEAKER_05"}, {"end": 7635.802, "text": "Okay.", "start": 7635.062, "speaker": "SPEAKER_05"}, {"end": 7642.165, "text": "So here's a nice example of adjusting for context without denying people information.", "start": 7635.802, "speaker": "SPEAKER_05"}, {"end": 7647.87, "text": "So if you look at this children's museum website, let me back it up and do it again.", "start": 7643.248, "speaker": "SPEAKER_05"}, {"end": 7654.072, "text": "You can see that they have up front this big visit us, call us, and find us, and hours open information.", "start": 7647.87, "speaker": "SPEAKER_05"}, {"end": 7656.432, "text": "There's still a menu in the upper right.", "start": 7654.072, "speaker": "SPEAKER_05"}, {"end": 7660.074, "text": "And the menu in the upper right, if you tap that, will get you to all their information.", "start": 7656.432, "speaker": "SPEAKER_05"}, {"end": 7664.015, "text": "You can go see the exhibits, education stuff, birthday parties.", "start": 7660.074, "speaker": "SPEAKER_05"}, {"end": 7672.238, "text": "Now when this same website is displayed on a larger screen, you can see contact us, find us, and the hours kind of get reduced in visual presence.", "start": 7664.015, "speaker": "SPEAKER_05"}, {"end": 7678.653, "text": "They're still there, they haven't cut any information away, but they change the priority of information.", "start": 7673.05, "speaker": "SPEAKER_05"}, {"end": 7686.358, "text": "So when you're on this smaller screen, chances are more likely that you're potentially trying to get there, right?", "start": 7678.653, "speaker": "SPEAKER_05"}, {"end": 7699.385, "text": "And so therefore they prioritize that info, but they still have the ability for you to fart around on the couch and see the specials, look at their education programs, explore the content, and you kind of go this way.", "start": 7686.358, "speaker": "SPEAKER_05"}, {"end": 7705.979, "text": "So the question always comes up, well, don't people use these devices in different ways?", "start": 7701.315, "speaker": "SPEAKER_05"}, {"end": 7709.222, "text": "And if you look at device usage, I've got a bunch of slides.", "start": 7705.979, "speaker": "SPEAKER_05"}, {"end": 7713.126, "text": "These are sort of older ones, but I think the message is still the same.", "start": 7709.222, "speaker": "SPEAKER_05"}, {"end": 7718.03, "text": "Desktop usage tends to spike during the middle of the day when people are at work.", "start": 7713.126, "speaker": "SPEAKER_05"}, {"end": 7719.591, "text": "And again, they're farting around.", "start": 7718.03, "speaker": "SPEAKER_05"}, {"end": 7723.114, "text": "So you'll pretty much see this pattern all the time for desktop.", "start": 7719.591, "speaker": "SPEAKER_05"}, {"end": 7726.598, "text": "Tablets, in general, have kind of an evening spike.", "start": 7723.114, "speaker": "SPEAKER_05"}, {"end": 7736.791, "text": "So like the morning spike and the evening spike happens on tablets, usually more of an evening spike in downtime.", "start": 7727.468, "speaker": "SPEAKER_05"}, {"end": 7744.514, "text": "This is now specific to Financial Times users, and you can see smartphone behavior happens throughout the day, but here there's a big spike.", "start": 7736.791, "speaker": "SPEAKER_05"}, {"end": 7748.27, "text": "And for the financial times, maybe it's people on their commute.", "start": 7745.548, "speaker": "SPEAKER_05"}, {"end": 7751.092, "text": "Maybe it's people checking in on the markets as they open.", "start": 7748.27, "speaker": "SPEAKER_05"}, {"end": 7753.514, "text": "Maybe there's a flurry of activity right there.", "start": 7751.092, "speaker": "SPEAKER_05"}, {"end": 7757.297, "text": "Maybe people are grabbing their phone and seeing what's happening in the morning.", "start": 7753.514, "speaker": "SPEAKER_05"}, {"end": 7760.839, "text": "And then in the evening, jumping back on smartphones.", "start": 7757.297, "speaker": "SPEAKER_05"}, {"end": 7763.121, "text": "This is BBC on the Olympics.", "start": 7760.839, "speaker": "SPEAKER_05"}, {"end": 7764.983, "text": "You see the same desktop spike.", "start": 7763.121, "speaker": "SPEAKER_05"}, {"end": 7771.007, "text": "You see mobile's pretty big all the time, and then fills in the space on when desktop's gone.", "start": 7764.983, "speaker": "SPEAKER_05"}, {"end": 7773.449, "text": "And you can see tablet again has this evening spike.", "start": 7771.007, "speaker": "SPEAKER_05"}, {"end": 7785.488, "text": "And I think this quote kind of sums it up pretty nicely, which is PC and tablet usage, desktop and tablet usage have really big spikes, but mobile is sort of anywhere and everywhere.", "start": 7774.6, "speaker": "SPEAKER_05"}, {"end": 7788.49, "text": "So it's always on in many ways.", "start": 7785.488, "speaker": "SPEAKER_05"}, {"end": 7797.176, "text": "And while we try to think about context, when we think where do mobile searches happen, most of them don't happen when people are actually mobile.", "start": 7788.49, "speaker": "SPEAKER_05"}, {"end": 7798.477, "text": "They happen at home and at work.", "start": 7797.176, "speaker": "SPEAKER_05"}, {"end": 7809.377, "text": "So while inferring context based on what kind of device and where people are can sometimes help us, many times we can actually be wrong.", "start": 7800.25, "speaker": "SPEAKER_05"}, {"end": 7819.105, "text": "And frankly, I just don't believe right now we have enough information to really dramatically change experiences based on what we think somebody is doing or isn't doing.", "start": 7809.377, "speaker": "SPEAKER_05"}, {"end": 7820.626, "text": "We're getting better, for sure.", "start": 7819.105, "speaker": "SPEAKER_05"}, {"end": 7822.367, "text": "We're getting more signals.", "start": 7820.626, "speaker": "SPEAKER_05"}, {"end": 7828.372, "text": "But it's not at that kind of, oh, yeah, that must be what this person is doing state.", "start": 7822.367, "speaker": "SPEAKER_05"}, {"end": 7830.459, "text": "Cool?", "start": 7829.579, "speaker": "SPEAKER_05"}, {"end": 7831.58, "text": "OK, where should it go?", "start": 7830.459, "speaker": "SPEAKER_05"}, {"end": 7840.082, "text": "I'm going to throw it to the center there.", "start": 7831.58, "speaker": "SPEAKER_05"}, {"end": 7840.622, "text": "Nice.", "start": 7840.082, "speaker": "SPEAKER_05"}, {"end": 7841.442, "text": "I think that was nice.", "start": 7840.622, "speaker": "SPEAKER_05"}, {"end": 7842.542, "text": "I couldn't tell.", "start": 7841.442, "speaker": "SPEAKER_05"}, {"end": 7844.323, "text": "I just heard the.", "start": 7842.542, "speaker": "SPEAKER_05"}, {"end": 7848.904, "text": "Yeah, I think he's fine.", "start": 7844.323, "speaker": "SPEAKER_12"}, {"end": 7849.264, "text": "Excellent.", "start": 7848.904, "speaker": "SPEAKER_05"}, {"end": 7861.193, "text": "So you encourage us to use this more natural way to, for example, log in or register.", "start": 7850.345, "speaker": "SPEAKER_12"}, {"end": 7866.437, "text": "And generally use natural gestures in our products.", "start": 7861.193, "speaker": "SPEAKER_12"}, {"end": 7870.3, "text": "But what if our customers use multiple devices?", "start": 7866.437, "speaker": "SPEAKER_12"}, {"end": 7883.032, "text": "So how to make iOS, Android app, and desktop experience compatible.", "start": 7871.341, "speaker": "SPEAKER_12"}, {"end": 7888.557, "text": "Yeah, I think there's all kinds of really useful ways to start thinking about multiple devices.", "start": 7883.032, "speaker": "SPEAKER_05"}, {"end": 7894.183, "text": "And this data also is a little older, but I think it'll...", "start": 7888.557, "speaker": "SPEAKER_05"}, {"end": 7896.99, "text": "give you a sense of how much people do it.", "start": 7895.189, "speaker": "SPEAKER_05"}, {"end": 7900.931, "text": "So in the US, half the people with a laptop also have a smartphone.", "start": 7896.99, "speaker": "SPEAKER_05"}, {"end": 7903.352, "text": "A third of smartphone users also have a tablet.", "start": 7900.931, "speaker": "SPEAKER_05"}, {"end": 7905.133, "text": "13% have all of them.", "start": 7903.352, "speaker": "SPEAKER_05"}, {"end": 7908.234, "text": "So there's a lot of cross-device kind of usage.", "start": 7905.133, "speaker": "SPEAKER_05"}, {"end": 7913.418, "text": "And 90% of people that have More than one device will use devices sequentially.", "start": 7908.234, "speaker": "SPEAKER_05"}, {"end": 7918.142, "text": "They'll go from the phone to the laptop, to the desktop, back to the laptop.", "start": 7913.418, "speaker": "SPEAKER_05"}, {"end": 7924.869, "text": "But then also, a huge amount of them do it not sequentially, but simultaneously.", "start": 7918.142, "speaker": "SPEAKER_05"}, {"end": 7928.032, "text": "So here's the amount of people that use these devices simultaneously.", "start": 7924.869, "speaker": "SPEAKER_05"}, {"end": 7932.136, "text": "While the TV's on, they're on their smartphone, or they've got their laptop.", "start": 7928.032, "speaker": "SPEAKER_05"}, {"end": 7935.078, "text": "on their lap while the TV's on.", "start": 7932.937, "speaker": "SPEAKER_05"}, {"end": 7940.441, "text": "And regardless of whether or not these numbers are still accurate today, the general pattern is still accurate.", "start": 7935.078, "speaker": "SPEAKER_05"}, {"end": 7944.683, "text": "People will move between devices, although use devices at the same time.", "start": 7940.441, "speaker": "SPEAKER_05"}, {"end": 7948.724, "text": "Which leads us to think about how can we design for cross-device usage, right?", "start": 7944.683, "speaker": "SPEAKER_05"}, {"end": 7952.886, "text": "How can we think about experiences that take advantage of this?", "start": 7948.724, "speaker": "SPEAKER_05"}, {"end": 7957.509, "text": "And there's a couple patterns to think through here that may give you some inspiration.", "start": 7952.886, "speaker": "SPEAKER_05"}, {"end": 7960.851, "text": "So one is this notion of access.", "start": 7958.509, "speaker": "SPEAKER_05"}, {"end": 7969.197, "text": "And access is simply making sure that what you have is available across these different devices in the appropriate way.", "start": 7960.851, "speaker": "SPEAKER_05"}, {"end": 7975.402, "text": "And the reason why this is becoming increasingly important is because people can transition between these devices much more seamlessly.", "start": 7969.197, "speaker": "SPEAKER_05"}, {"end": 7988.251, "text": "So for example, if I'm on my laptop and I go to my website, I click on this link, I go click on this other link, I click on yet another link, and I just navigate it around this thing.", "start": 7976.202, "speaker": "SPEAKER_05"}, {"end": 7997.578, "text": "Then when I fire up my mobile phone, you can see here with Chrome, we're actually showing you that laptop where you were just at and how long ago we synced.", "start": 7988.251, "speaker": "SPEAKER_05"}, {"end": 8004.823, "text": "So you can not only tap on that page and be exactly where you left off, but the back button actually works.", "start": 7997.578, "speaker": "SPEAKER_05"}, {"end": 8012.773, "text": "And you can go and follow that whole path that you just took on your desktop on your phone.", "start": 8005.906, "speaker": "SPEAKER_05"}, {"end": 8019.72, "text": "And so making sure that you have an experience that's available across these different devices makes these experiences work.", "start": 8012.773, "speaker": "SPEAKER_05"}, {"end": 8025.887, "text": "If you have like a separate mobile site or a separate tablet site, the URLs don't align, this kind of stuff can break.", "start": 8019.72, "speaker": "SPEAKER_05"}, {"end": 8030.423, "text": "Another example is flow.", "start": 8027.701, "speaker": "SPEAKER_05"}, {"end": 8035.646, "text": "So content, that's probably not the only thing we want to move seamlessly between our devices.", "start": 8030.423, "speaker": "SPEAKER_05"}, {"end": 8039.108, "text": "Processes should be able to move pretty seamlessly as well.", "start": 8035.646, "speaker": "SPEAKER_05"}, {"end": 8044.252, "text": "So with this, if you're in the middle of a task, you can just pick up right where you left off when switching tasks.", "start": 8039.108, "speaker": "SPEAKER_05"}, {"end": 8047.002, "text": "Again, I'm on my laptop.", "start": 8045.161, "speaker": "SPEAKER_05"}, {"end": 8050.463, "text": "I search for a destination so I know where I'm traveling.", "start": 8047.002, "speaker": "SPEAKER_05"}, {"end": 8059.507, "text": "When I open up my Maps app, in this case Google Maps, and I tap search, there's that location I just looked for and I can take off running.", "start": 8050.463, "speaker": "SPEAKER_05"}, {"end": 8064.47, "text": "So you can just pick up your flow right where you left off.", "start": 8059.507, "speaker": "SPEAKER_05"}, {"end": 8069.932, "text": "Control is another interesting pattern where one device controls another, often remotely.", "start": 8064.47, "speaker": "SPEAKER_05"}, {"end": 8075.794, "text": "And so here we built a, oh, actually I thought I had a video of this.", "start": 8071.09, "speaker": "SPEAKER_05"}, {"end": 8076.334, "text": "Is this the video?", "start": 8075.794, "speaker": "SPEAKER_05"}, {"end": 8079.497, "text": "Let's see, maybe I just skipped through it.", "start": 8076.334, "speaker": "SPEAKER_05"}, {"end": 8093.288, "text": "So we built an experience with Microsoft a while ago where you can sort of talk to Xbox and you can connect your phone to a big screen and you can see I can actually be voting on things.", "start": 8079.497, "speaker": "SPEAKER_05"}, {"end": 8097.086, "text": "as they're displayed up on the TV.", "start": 8095.224, "speaker": "SPEAKER_05"}, {"end": 8102.53, "text": "And the TV interface has a very different UI than what I have on the desktop.", "start": 8097.086, "speaker": "SPEAKER_05"}, {"end": 8105.873, "text": "And I can watch TV in the one screen.", "start": 8102.53, "speaker": "SPEAKER_05"}, {"end": 8114.02, "text": "And on the other screen, I can see while I'm voting here, I can go and do a search for Futurama.", "start": 8105.873, "speaker": "SPEAKER_05"}, {"end": 8117.603, "text": "And now all the Futurama questions pop up there, and I can interact.", "start": 8114.02, "speaker": "SPEAKER_05"}, {"end": 8119.084, "text": "But you get the idea here, right?", "start": 8117.603, "speaker": "SPEAKER_05"}, {"end": 8122.667, "text": "One device is controlling either the other device or part of the other device.", "start": 8119.084, "speaker": "SPEAKER_05"}, {"end": 8132.148, "text": "And the last one is a push, and this is the one I think is actually most interesting and perhaps underutilized.", "start": 8123.983, "speaker": "SPEAKER_05"}, {"end": 8133.829, "text": "So eBay has a nice example of this.", "start": 8132.148, "speaker": "SPEAKER_05"}, {"end": 8140.953, "text": "If I'm on my laptop and I'm listing an item for sale, I can say I'm going to grab an image from my phone.", "start": 8133.829, "speaker": "SPEAKER_05"}, {"end": 8143.154, "text": "A little notification pops up on my phone.", "start": 8140.953, "speaker": "SPEAKER_05"}, {"end": 8146.516, "text": "It opens the eBay app and jumps right to the camera roll.", "start": 8143.154, "speaker": "SPEAKER_05"}, {"end": 8154.707, "text": "And so I can hit the camera roll, and I can use my phone, which is always in my pocket, to take a picture of the thing I'm selling.", "start": 8147.365, "speaker": "SPEAKER_05"}, {"end": 8157.408, "text": "So I just hit that, take a picture.", "start": 8154.707, "speaker": "SPEAKER_05"}, {"end": 8161.109, "text": "It gets saved.", "start": 8157.408, "speaker": "SPEAKER_05"}, {"end": 8161.61, "text": "I hit Done.", "start": 8161.109, "speaker": "SPEAKER_05"}, {"end": 8168.952, "text": "And then automatically, from having taken that picture, it will now instantly show up on the desktop without me doing anything.", "start": 8161.61, "speaker": "SPEAKER_05"}, {"end": 8173.444, "text": "So it's a really nice example of these devices working together.", "start": 8170.12, "speaker": "SPEAKER_05"}, {"end": 8176.468, "text": "This has a camera that's always likely with me.", "start": 8173.444, "speaker": "SPEAKER_05"}, {"end": 8179.652, "text": "Getting pictures onto the laptop is kind of hard.", "start": 8176.468, "speaker": "SPEAKER_05"}, {"end": 8181.073, "text": "It's sort of a pain in the butt.", "start": 8179.652, "speaker": "SPEAKER_05"}, {"end": 8187.922, "text": "So having this seamless integration through a push notification, it gives me an opportunity to use those devices together.", "start": 8181.073, "speaker": "SPEAKER_05"}, {"end": 8193.657, "text": "So hopefully that gives you some ideas on how we can think cross-device.", "start": 8189.895, "speaker": "SPEAKER_05"}, {"end": 8198.28, "text": "Again, the general patterns are make sure your stuff's accessible.", "start": 8193.657, "speaker": "SPEAKER_05"}, {"end": 8201.621, "text": "Responsive design is a great tool for that on the web.", "start": 8198.28, "speaker": "SPEAKER_05"}, {"end": 8207.625, "text": "We can talk a little bit about how to make a consistent experience across native platforms as well.", "start": 8201.621, "speaker": "SPEAKER_05"}, {"end": 8211.396, "text": "Flow is this notion of processes.", "start": 8208.655, "speaker": "SPEAKER_05"}, {"end": 8214.397, "text": "Control gives you a chance to have one device control the other.", "start": 8211.396, "speaker": "SPEAKER_05"}, {"end": 8216.138, "text": "That's potentially more rare.", "start": 8214.397, "speaker": "SPEAKER_05"}, {"end": 8229.322, "text": "But this push scenario, I think, is the most exciting one, where you can use the best part of each device together, cumulatively, to create a overall experience that works well.", "start": 8216.138, "speaker": "SPEAKER_05"}, {"end": 8230.722, "text": "OK.", "start": 8229.322, "speaker": "SPEAKER_05"}, {"end": 8231.363, "text": "Answer your question?", "start": 8230.722, "speaker": "SPEAKER_05"}, {"end": 8232.843, "text": "OK.", "start": 8231.363, "speaker": "SPEAKER_05"}, {"end": 8233.763, "text": "Three thumbs up today.", "start": 8232.843, "speaker": "SPEAKER_05"}, {"end": 8236.564, "text": "I'm doing awesome.", "start": 8233.763, "speaker": "SPEAKER_05"}, {"end": 8237.385, "text": "Where should we throw it?", "start": 8236.564, "speaker": "SPEAKER_05"}, {"end": 8238.345, "text": "Looks like right behind you.", "start": 8237.385, "speaker": "SPEAKER_05"}, {"end": 8241.404, "text": "Easy throw, yes.", "start": 8239.803, "speaker": "SPEAKER_05"}, {"end": 8242.025, "text": "Hi.", "start": 8241.404, "speaker": "SPEAKER_07"}, {"end": 8243.386, "text": "Hi.", "start": 8242.025, "speaker": "SPEAKER_07"}, {"end": 8248.371, "text": "So you talked a lot about the checkout process, how to make it simpler and smoother.", "start": 8243.386, "speaker": "SPEAKER_07"}, {"end": 8253.095, "text": "Do you have any tips for what happens before the checkout process?", "start": 8248.371, "speaker": "SPEAKER_07"}, {"end": 8261.642, "text": "So for example, making it simpler to choose from products or boosting the motivation to buy and so on?", "start": 8253.095, "speaker": "SPEAKER_07"}, {"end": 8264.525, "text": "Yeah, so in terms of like search patterns and the like?", "start": 8261.642, "speaker": "SPEAKER_05"}, {"end": 8270.89, "text": "There's a whole series of things we can discuss around that.", "start": 8266.127, "speaker": "SPEAKER_05"}, {"end": 8280.075, "text": "I'm trying to think what's the fastest and easiest way we can talk about it.", "start": 8270.89, "speaker": "SPEAKER_05"}, {"end": 8281.716, "text": "Let me scan for half a second.", "start": 8280.075, "speaker": "SPEAKER_05"}, {"end": 8289.7, "text": "I'll look at one thing here and see if it helps us.", "start": 8281.716, "speaker": "SPEAKER_05"}, {"end": 8296.277, "text": "Oops, that's the whole talk I showed this morning.", "start": 8293.915, "speaker": "SPEAKER_05"}, {"end": 8298.899, "text": "We don't want to reference that again.", "start": 8296.277, "speaker": "SPEAKER_05"}, {"end": 8300.121, "text": "OK, so here we go.", "start": 8298.899, "speaker": "SPEAKER_05"}, {"end": 8306.666, "text": "One more deck open recent.", "start": 8300.121, "speaker": "SPEAKER_05"}, {"end": 8310.57, "text": "OK.", "start": 8306.666, "speaker": "SPEAKER_05"}, {"end": 8317.156, "text": "So I think there's a lot of opportunities that we have to improve overall browsing experiences in a number of different ways.", "start": 8310.57, "speaker": "SPEAKER_05"}, {"end": 8325.43, "text": "Right now, we're kind of dependent on search and browse as our primary mechanisms.", "start": 8318.523, "speaker": "SPEAKER_05"}, {"end": 8336.501, "text": "But for instance, whenever you go and represent any kind of like hierarchical list for people to find things, there's an opportunity to actually get them invested in what's inside of each category.", "start": 8325.43, "speaker": "SPEAKER_05"}, {"end": 8339.369, "text": "to help them make their decisions appropriately.", "start": 8337.368, "speaker": "SPEAKER_05"}, {"end": 8344.032, "text": "So for example, this doesn't necessarily apply to shopping, because this is in the finance space.", "start": 8339.369, "speaker": "SPEAKER_05"}, {"end": 8348.374, "text": "But contrast that browse path on the left with this one on the right.", "start": 8344.032, "speaker": "SPEAKER_05"}, {"end": 8350.195, "text": "You'll have a sense of what's inside.", "start": 8348.374, "speaker": "SPEAKER_05"}, {"end": 8351.756, "text": "You have a sense of some of the activity.", "start": 8350.195, "speaker": "SPEAKER_05"}, {"end": 8354.158, "text": "And you know why you should access any of them.", "start": 8351.756, "speaker": "SPEAKER_05"}, {"end": 8365.728, "text": "And then here, this is sort of the difference between kind of a static browse a category list and what you can do to make that a lot more dynamic and engaging.", "start": 8355.278, "speaker": "SPEAKER_05"}, {"end": 8374.136, "text": "So here in the books department, not only is there search within a category, there's actually merchandising that drives you to popular stuff.", "start": 8365.728, "speaker": "SPEAKER_05"}, {"end": 8376.298, "text": "There's this primary nav, this sub-nav.", "start": 8374.136, "speaker": "SPEAKER_05"}, {"end": 8386.34, "text": "And so introducing these kinds of elements to any kind of guided browsing path can give people clues on where they should or shouldn't go or what they could do.", "start": 8376.918, "speaker": "SPEAKER_05"}, {"end": 8390.801, "text": "So I think those are a couple things to do for sure.", "start": 8386.34, "speaker": "SPEAKER_05"}, {"end": 8397.063, "text": "In terms of increasing motivation on view item pages, I feel like you guys are doing enough already.", "start": 8390.801, "speaker": "SPEAKER_05"}, {"end": 8400.124, "text": "It's just me.", "start": 8397.063, "speaker": "SPEAKER_05"}, {"end": 8403.565, "text": "There's like 15 million things that tell me I should buy the darn thing.", "start": 8400.124, "speaker": "SPEAKER_05"}, {"end": 8412.318, "text": "So I don't necessarily have too much on that.", "start": 8406.474, "speaker": "SPEAKER_05"}, {"end": 8412.578, "text": "Next.", "start": 8412.318, "speaker": "SPEAKER_05"}, {"end": 8415.68, "text": "OK, you've got to really throw it down.", "start": 8412.578, "speaker": "SPEAKER_05"}, {"end": 8417.18, "text": "There's like four down here.", "start": 8415.68, "speaker": "SPEAKER_05"}, {"end": 8418.861, "text": "Ready?", "start": 8417.18, "speaker": "SPEAKER_05"}, {"end": 8419.322, "text": "Incoming.", "start": 8418.861, "speaker": "SPEAKER_05"}, {"end": 8420.643, "text": "Whoa!", "start": 8419.322, "speaker": "SPEAKER_05"}, {"end": 8421.643, "text": "Oh!", "start": 8420.643, "speaker": "SPEAKER_05"}, {"end": 8425.325, "text": "That was almost perfect.", "start": 8421.643, "speaker": "SPEAKER_05"}, {"end": 8430.418, "text": "So actually, It is related to the last question.", "start": 8425.325, "speaker": "SPEAKER_10"}, {"end": 8435.683, "text": "We are really struggling on our webshop with navigation.", "start": 8430.418, "speaker": "SPEAKER_10"}, {"end": 8443.51, "text": "We have this old-fashioned hamburger menu.", "start": 8435.683, "speaker": "SPEAKER_10"}, {"end": 8447.113, "text": "I thought about it when you showed the slides about the reach of the thumb.", "start": 8443.51, "speaker": "SPEAKER_10"}, {"end": 8459.234, "text": "So a lot of web shops and also our web shop still have the hamburger on top, far away from the thumb actually, which is quite weird to my opinion.", "start": 8448.884, "speaker": "SPEAKER_10"}, {"end": 8473.147, "text": "But I'm wondering about, I'm really curious about what opinions you have about optimal navigation on web shops that sell a lot of categories and if we should move it to down or", "start": 8459.234, "speaker": "SPEAKER_10"}, {"end": 8476.16, "text": "Other ideas?", "start": 8474.399, "speaker": "SPEAKER_05"}, {"end": 8476.961, "text": "OK.", "start": 8476.16, "speaker": "SPEAKER_05"}, {"end": 8479.302, "text": "So let's talk about that for half a second.", "start": 8476.961, "speaker": "SPEAKER_05"}, {"end": 8489.788, "text": "So the hamburger menu pattern, for those of you that don't know, it's kind of you hit a little element in the upper left, and you go to the menu on the side.", "start": 8479.302, "speaker": "SPEAKER_05"}, {"end": 8493.07, "text": "So here you can go and hit the hamburger menu.", "start": 8489.788, "speaker": "SPEAKER_05"}, {"end": 8494.971, "text": "You see some navigation.", "start": 8493.07, "speaker": "SPEAKER_05"}, {"end": 8495.871, "text": "You hit it back.", "start": 8494.971, "speaker": "SPEAKER_05"}, {"end": 8496.412, "text": "It goes away.", "start": 8495.871, "speaker": "SPEAKER_05"}, {"end": 8500.185, "text": "It looks like that, and here's another example.", "start": 8497.363, "speaker": "SPEAKER_05"}, {"end": 8504.168, "text": "This one over on ESPN, you hit it, it comes back, goes away.", "start": 8500.185, "speaker": "SPEAKER_05"}, {"end": 8511.253, "text": "And this used to be the case on Facebook a long time ago, which is up here.", "start": 8504.168, "speaker": "SPEAKER_05"}, {"end": 8515.676, "text": "In fact, the hamburger menu has become so popular, it has its own Twitter account called Mr.", "start": 8511.253, "speaker": "SPEAKER_05"}, {"end": 8516.076, "text": "Liney.", "start": 8515.676, "speaker": "SPEAKER_05"}, {"end": 8522.621, "text": "And he basically goes around and tweeting all the places that he gets used because it gives him a nice ego boost.", "start": 8516.076, "speaker": "SPEAKER_05"}, {"end": 8527.536, "text": "Now, there's good reasons for him to be happy about this.", "start": 8524.475, "speaker": "SPEAKER_05"}, {"end": 8531.637, "text": "If you go and look around, you see hamburger menus everywhere.", "start": 8527.536, "speaker": "SPEAKER_05"}, {"end": 8533.077, "text": "They're all over the place.", "start": 8531.637, "speaker": "SPEAKER_05"}, {"end": 8541.92, "text": "And when you talk to people about why are they using hamburger menus, you generally hear people say something like, oh, everybody's doing it, so I will too.", "start": 8533.077, "speaker": "SPEAKER_05"}, {"end": 8545.921, "text": "Or you hear Facebook is using it, so I will too.", "start": 8541.92, "speaker": "SPEAKER_05"}, {"end": 8552.703, "text": "And in particular, the second one, I think, is very, very dangerous because you don't know why they're doing it.", "start": 8545.921, "speaker": "SPEAKER_05"}, {"end": 8561.337, "text": "I for years have gone and talked at companies where I've been really terrified going in there because I feel like they know everything.", "start": 8553.968, "speaker": "SPEAKER_05"}, {"end": 8562.478, "text": "Oh my God, it's eBay.", "start": 8561.337, "speaker": "SPEAKER_05"}, {"end": 8564.721, "text": "They know so much about mobile commerce.", "start": 8562.478, "speaker": "SPEAKER_05"}, {"end": 8569.406, "text": "And when you come in, you realize they're figuring stuff out just like the rest of us.", "start": 8564.721, "speaker": "SPEAKER_05"}, {"end": 8585.056, "text": "And in particular, the case of Facebook, though they had this hamburger menu, When they did a redesign, they found that engagement, satisfaction, revenue, perception of speed, everything they cared about, actually did a lot better when they moved to this.", "start": 8569.406, "speaker": "SPEAKER_05"}, {"end": 8594.165, "text": "And the reason why they ended up sticking with that hamburger menu for so long is because their app was really damn slow and buggy and it used memory.", "start": 8586.277, "speaker": "SPEAKER_05"}, {"end": 8600.392, "text": "It was essentially JavaScript creating a pseudo web browser inside of a native app inside of the OS.", "start": 8594.165, "speaker": "SPEAKER_05"}, {"end": 8605.557, "text": "So they had a one or two star review for a long time because it was so crappy and slow.", "start": 8600.392, "speaker": "SPEAKER_05"}, {"end": 8609.461, "text": "So they were spending all their time rewriting the app and not thinking about the nav.", "start": 8605.557, "speaker": "SPEAKER_05"}, {"end": 8618.98, "text": "When they implemented an A-B testing system and they could compare the nav against the other, they realized, hey, this actually works a lot better.", "start": 8610.355, "speaker": "SPEAKER_05"}, {"end": 8625.123, "text": "So be careful when you look at what other people are doing and assume reasons why it is that way.", "start": 8618.98, "speaker": "SPEAKER_05"}, {"end": 8627.684, "text": "It could be God knows what reason.", "start": 8625.123, "speaker": "SPEAKER_05"}, {"end": 8630.465, "text": "You know why the hamburger menu is on that person's site?", "start": 8627.684, "speaker": "SPEAKER_05"}, {"end": 8633.227, "text": "Because Bob the developer and Jane the PM hate each other.", "start": 8630.465, "speaker": "SPEAKER_05"}, {"end": 8638.29, "text": "And Bob refuses to implement the new now.", "start": 8635.448, "speaker": "SPEAKER_05"}, {"end": 8642.974, "text": "And Jane escalated it to her boss, and Bob's being such an a-hole.", "start": 8638.29, "speaker": "SPEAKER_05"}, {"end": 8643.554, "text": "You don't know.", "start": 8642.974, "speaker": "SPEAKER_05"}, {"end": 8644.995, "text": "Human beings are human beings.", "start": 8643.554, "speaker": "SPEAKER_05"}, {"end": 8647.297, "text": "There could be a gazillion reasons.", "start": 8644.995, "speaker": "SPEAKER_05"}, {"end": 8650.9, "text": "So it's always best to go and get your own opinion on things.", "start": 8647.297, "speaker": "SPEAKER_05"}, {"end": 8655.303, "text": "And I went and talked to a bunch of people around the hamburger menu.", "start": 8650.9, "speaker": "SPEAKER_05"}, {"end": 8657.665, "text": "Here, some people got it, thought it was an Apple thing.", "start": 8655.303, "speaker": "SPEAKER_05"}, {"end": 8661.968, "text": "Poor Danny at AT&T found people have no idea what they are.", "start": 8657.665, "speaker": "SPEAKER_05"}, {"end": 8663.209, "text": "That's a shame.", "start": 8661.968, "speaker": "SPEAKER_05"}, {"end": 8670.294, "text": "And then you can learn more through these qualitative things, talking to more people.", "start": 8664.472, "speaker": "SPEAKER_05"}, {"end": 8672.255, "text": "You can run quantitative tests.", "start": 8670.294, "speaker": "SPEAKER_05"}, {"end": 8685.579, "text": "James Foster ran this quant test where he took the hamburger menu, compared it against the hamburger of the word menu, compared it against the hamburger inside of a button type thing, and found adding the word menu helps people understand it.", "start": 8672.255, "speaker": "SPEAKER_05"}, {"end": 8690.601, "text": "But making it look like something you can actually tap helps more people understand it.", "start": 8685.579, "speaker": "SPEAKER_05"}, {"end": 8691.581, "text": "Shocking, I know.", "start": 8690.601, "speaker": "SPEAKER_05"}, {"end": 8699.048, "text": "Then he took the winner of round one, and he tested it against the Word menu.", "start": 8693.723, "speaker": "SPEAKER_05"}, {"end": 8704.432, "text": "He tested it against the Word menu with an icon, and he tested it with just the Word menu.", "start": 8699.048, "speaker": "SPEAKER_05"}, {"end": 8710.777, "text": "And what he found was using words instead of cryptic symbols tends to improve things.", "start": 8704.432, "speaker": "SPEAKER_05"}, {"end": 8712.959, "text": "Shocking, I know.", "start": 8710.777, "speaker": "SPEAKER_05"}, {"end": 8716.342, "text": "He also found that having the icon in the menu doesn't do that much.", "start": 8712.959, "speaker": "SPEAKER_05"}, {"end": 8722.207, "text": "And this one is the most funny to me, because it turns out like 22% of people need a buttony thing to understand something's tappable.", "start": 8716.342, "speaker": "SPEAKER_05"}, {"end": 8726.635, "text": "Because before, remember when you put the buttony thing, 22% more used it.", "start": 8723.152, "speaker": "SPEAKER_05"}, {"end": 8734.262, "text": "When you took away the buttony thing, those poor 22% of people were like, oh my god.", "start": 8726.635, "speaker": "SPEAKER_05"}, {"end": 8737.965, "text": "So then he went and ran a grand finale.", "start": 8734.262, "speaker": "SPEAKER_05"}, {"end": 8747.873, "text": "He took the hamburger menu in the buttony thing, the word menu in the buttony thing, and he ran it against almost 250,000 unique mobile visitors.", "start": 8737.965, "speaker": "SPEAKER_05"}, {"end": 8749.775, "text": "And it turned out the word menu outperformed.", "start": 8747.873, "speaker": "SPEAKER_05"}, {"end": 8753.073, "text": "which is cool.", "start": 8751.092, "speaker": "SPEAKER_05"}, {"end": 8757.034, "text": "Now, of course, all of this matters your specific context.", "start": 8753.073, "speaker": "SPEAKER_05"}, {"end": 8760.416, "text": "And I'm going to reference Booking.com because they brought this up.", "start": 8757.034, "speaker": "SPEAKER_05"}, {"end": 8763.717, "text": "And I think somebody, maybe it was Craig, quoted from them.", "start": 8760.416, "speaker": "SPEAKER_05"}, {"end": 8769.579, "text": "When they ran this test, they found no significant impact between hamburger menu and the word menu.", "start": 8763.717, "speaker": "SPEAKER_05"}, {"end": 8774.341, "text": "But it turns out, what is behind the hamburger menu on Booking.com?", "start": 8769.579, "speaker": "SPEAKER_05"}, {"end": 8777.642, "text": "The ability to change the currency in which your results apply.", "start": 8774.341, "speaker": "SPEAKER_05"}, {"end": 8783.098, "text": "your ability to create a profile.", "start": 8779.835, "speaker": "SPEAKER_05"}, {"end": 8786.361, "text": "In other words, this is not critical stuff.", "start": 8783.098, "speaker": "SPEAKER_05"}, {"end": 8792.827, "text": "And so in their specific user base, it may or may not matter that people even access that menu.", "start": 8786.361, "speaker": "SPEAKER_05"}, {"end": 8796.51, "text": "How often are you changing your currency settings?", "start": 8792.827, "speaker": "SPEAKER_05"}, {"end": 8799.959, "text": "Hm, doesn't matter so much.", "start": 8797.738, "speaker": "SPEAKER_05"}, {"end": 8802.961, "text": "In other situations, and by the way, I think if they had run this test, it'd be different.", "start": 8799.959, "speaker": "SPEAKER_05"}, {"end": 8806.783, "text": "But anyway, in other situations, it matters a ton.", "start": 8802.961, "speaker": "SPEAKER_05"}, {"end": 8809.065, "text": "So let me skip this slide and look at this slide.", "start": 8806.783, "speaker": "SPEAKER_05"}, {"end": 8821.112, "text": "For a time, what they did when they moved to the hamburger menus, they took all the categories of their newspaper, world, politics, entertainment, sports, they hid them behind this tiny little thing.", "start": 8809.065, "speaker": "SPEAKER_05"}, {"end": 8824.934, "text": "And guess what happened?", "start": 8821.112, "speaker": "SPEAKER_05"}, {"end": 8825.795, "text": "To their metrics.", "start": 8824.934, "speaker": "SPEAKER_05"}, {"end": 8830.339, "text": "Because people didn't know where the sections were anymore.", "start": 8827.155, "speaker": "SPEAKER_05"}, {"end": 8831.261, "text": "Where's sports?", "start": 8830.339, "speaker": "SPEAKER_05"}, {"end": 8832.482, "text": "Where's news?", "start": 8831.261, "speaker": "SPEAKER_05"}, {"end": 8833.724, "text": "Where's entertainment?", "start": 8832.482, "speaker": "SPEAKER_05"}, {"end": 8837.97, "text": "And they got so desperate, they had to add the word menu.", "start": 8833.724, "speaker": "SPEAKER_05"}, {"end": 8839.773, "text": "And then they added this thing.", "start": 8837.97, "speaker": "SPEAKER_05"}, {"end": 8841.535, "text": "New time, new navigation.", "start": 8839.773, "speaker": "SPEAKER_05"}, {"end": 8842.436, "text": "Click the menu.", "start": 8841.535, "speaker": "SPEAKER_05"}, {"end": 8842.557, "text": "OK, OK.", "start": 8842.436, "speaker": "SPEAKER_05"}, {"end": 8848.381, "text": "Desperation, right?", "start": 8846.341, "speaker": "SPEAKER_05"}, {"end": 8853.242, "text": "You don't need the qualitative studies and the quantitative studies that I showed you a second ago sometimes.", "start": 8848.381, "speaker": "SPEAKER_05"}, {"end": 8856.183, "text": "You just need to see something like this, these Band-Aids.", "start": 8853.242, "speaker": "SPEAKER_05"}, {"end": 8861.264, "text": "Look for the Band-Aids, and that's where you'll find the ouchies, the usability ouchies.", "start": 8856.183, "speaker": "SPEAKER_05"}, {"end": 8868.145, "text": "And it's a nice illustration of we shouldn't just copy patterns, right?", "start": 8861.264, "speaker": "SPEAKER_05"}, {"end": 8872.166, "text": "To come back to this for half a second, where did I put this?", "start": 8868.145, "speaker": "SPEAKER_05"}, {"end": 8878.866, "text": "Because copying patterns, again, we don't know why things are the way they are.", "start": 8873.439, "speaker": "SPEAKER_05"}, {"end": 8887.136, "text": "So if we come back to this, this was hamburger menu, post-hamburger menu for TabBar on Facebook.", "start": 8878.866, "speaker": "SPEAKER_05"}, {"end": 8888.958, "text": "And you can see the before and after view.", "start": 8887.136, "speaker": "SPEAKER_05"}, {"end": 8891.761, "text": "And they had better engagement, satisfaction, what have you.", "start": 8888.958, "speaker": "SPEAKER_05"}, {"end": 8906.437, "text": "Another example, in this app redesign, they used to have a hamburger menu, they moved to the tab bar, session time increased 70%, 65% increase in daily active users.", "start": 8892.843, "speaker": "SPEAKER_05"}, {"end": 8909.26, "text": "And I should pause because this is super dramatic, right?", "start": 8906.437, "speaker": "SPEAKER_05"}, {"end": 8910.521, "text": "How is this even possible?", "start": 8909.26, "speaker": "SPEAKER_05"}, {"end": 8915.947, "text": "Well, what happens is, before, all of this is out of sight, out of mind.", "start": 8911.222, "speaker": "SPEAKER_05"}, {"end": 8924.176, "text": "And here, we're making it not only visible, but we're showing activity, telling you what's actually happening in this app, putting it front and center.", "start": 8915.947, "speaker": "SPEAKER_05"}, {"end": 8928.9, "text": "So you know there's things that are meaningful to you.", "start": 8924.176, "speaker": "SPEAKER_05"}, {"end": 8948.668, "text": "Sort of looking at it in the opposite direction, this app went from a tab design with a set of options around what's inside of this entertainment app to this navigation drawer pattern where they hid everything behind their icon and you have to slide it open to see it.", "start": 8930.554, "speaker": "SPEAKER_05"}, {"end": 8954.253, "text": "And when they did that and they looked at their analytics, they said it was a disaster.", "start": 8948.668, "speaker": "SPEAKER_05"}, {"end": 8956.975, "text": "Weekly and daily frequency was down.", "start": 8954.253, "speaker": "SPEAKER_05"}, {"end": 8962.787, "text": "In fact, weekly and Engagement time was halved.", "start": 8956.975, "speaker": "SPEAKER_05"}, {"end": 8963.968, "text": "Time spent in the app went down.", "start": 8962.787, "speaker": "SPEAKER_05"}, {"end": 8971.015, "text": "So they lost half of their activity when they buried these different features and functions, which is a huge amount.", "start": 8963.968, "speaker": "SPEAKER_05"}, {"end": 8975.564, "text": "So what does all of this stuff mean?", "start": 8973.623, "speaker": "SPEAKER_05"}, {"end": 8988.193, "text": "I think what it means is we want to make visible to people what is inside of our apps, why they should use them, how they should use them, and those kinds of things.", "start": 8975.564, "speaker": "SPEAKER_05"}, {"end": 8999.06, "text": "And last year, I won't repeat it this year because we don't have enough time, but last year at this conference, I walked through a whole redesign going from a hidden nav structure to a visible nav structure.", "start": 8988.193, "speaker": "SPEAKER_05"}, {"end": 9002.922, "text": "And I outlined what the big benefits of that were.", "start": 8999.06, "speaker": "SPEAKER_05"}, {"end": 9009.084, "text": "But what it basically boils down to is very, very similar to what you see inside this image here.", "start": 9002.922, "speaker": "SPEAKER_05"}, {"end": 9011.624, "text": "And I actually think I've got even more examples of this now.", "start": 9009.084, "speaker": "SPEAKER_05"}, {"end": 9016.045, "text": "It turns out examples of this keep popping up left and right.", "start": 9011.624, "speaker": "SPEAKER_05"}, {"end": 9019.526, "text": "Let me see if I can find this last one.", "start": 9016.045, "speaker": "SPEAKER_05"}, {"end": 9028.348, "text": "And so my advice around navigation is, first and foremost, understand the things that help people orient to what you do.", "start": 9019.526, "speaker": "SPEAKER_05"}, {"end": 9028.968, "text": "Here's the other one.", "start": 9028.348, "speaker": "SPEAKER_05"}, {"end": 9030.408, "text": "Let me play that.", "start": 9028.968, "speaker": "SPEAKER_05"}, {"end": 9047.722, "text": "So here's another example where they went from this hamburger menu down to this bottom navigation menu, double-digit growth in monthly active users, double-digit growth in visitors, triple-digit growth in users, what have you.", "start": 9033.833, "speaker": "SPEAKER_05"}, {"end": 9059.567, "text": "And so it's just front and center what the main pieces of the application are, what you can do there, and what you should do there, whereas before, everything is often hidden.", "start": 9047.722, "speaker": "SPEAKER_05"}, {"end": 9069.474, "text": "And I guess maybe I can dig up one chart of what I showed last year to tempt you to go look at that video, if you so choose.", "start": 9059.567, "speaker": "SPEAKER_05"}, {"end": 9071.235, "text": "Let me find it.", "start": 9069.474, "speaker": "SPEAKER_05"}, {"end": 9072.116, "text": "Here we go.", "start": 9071.235, "speaker": "SPEAKER_05"}, {"end": 9079.08, "text": "So when we put some things inside of the bottom navigation menu, here's kind of the chart we saw.", "start": 9072.116, "speaker": "SPEAKER_05"}, {"end": 9081.462, "text": "This little chart is the test of it.", "start": 9079.08, "speaker": "SPEAKER_05"}, {"end": 9086.98, "text": "And then that's the actual one of it.", "start": 9082.591, "speaker": "SPEAKER_05"}, {"end": 9091.41, "text": "And I have similar charts for all kinds of other important-y things.", "start": 9086.98, "speaker": "SPEAKER_05"}, {"end": 9095.448, "text": "Does that help answer that question?", "start": 9093.868, "speaker": "SPEAKER_05"}, {"end": 9105.15, "text": "So this is, again, I think comes down to the trick of designing for mobile, finding the right stuff to put on the small screen at the right time.", "start": 9095.448, "speaker": "SPEAKER_05"}, {"end": 9116.892, "text": "When people enter a service or when they're at the top level, one of the things that I think is the right thing to show them is the scope of what's possible and the means to get to that possibility.", "start": 9105.15, "speaker": "SPEAKER_05"}, {"end": 9124.514, "text": "And so navigational systems, which hide all of that possibility and get rid of it, do a disservice to people.", "start": 9116.892, "speaker": "SPEAKER_05"}, {"end": 9125.936, "text": "They don't know what's available.", "start": 9124.514, "speaker": "SPEAKER_05"}, {"end": 9127.517, "text": "Therefore, they don't get into it.", "start": 9125.936, "speaker": "SPEAKER_05"}, {"end": 9131.24, "text": "And as a result, you may end up losing them from an engagement perspective.", "start": 9127.517, "speaker": "SPEAKER_05"}, {"end": 9136.765, "text": "All right, I don't know where the mic is anymore.", "start": 9131.24, "speaker": "SPEAKER_05"}, {"end": 9139.127, "text": "Who's next?", "start": 9136.765, "speaker": "SPEAKER_05"}, {"end": 9139.728, "text": "Let's go up.", "start": 9139.127, "speaker": "SPEAKER_05"}, {"end": 9140.768, "text": "There you go.", "start": 9139.728, "speaker": "SPEAKER_05"}, {"end": 9141.709, "text": "Nice.", "start": 9140.768, "speaker": "SPEAKER_05"}, {"end": 9143.511, "text": "Hi.", "start": 9141.709, "speaker": "SPEAKER_08"}, {"end": 9153.408, "text": "In the previous session, you gave the example of a newly designed credit card form, which on first glance looked are pretty complex to handle.", "start": 9143.511, "speaker": "SPEAKER_08"}, {"end": 9156.169, "text": "So can you give a little more detail to that?", "start": 9153.408, "speaker": "SPEAKER_08"}, {"end": 9158.749, "text": "Sure, on that credit card pattern thing?", "start": 9156.169, "speaker": "SPEAKER_05"}, {"end": 9163.31, "text": "Yeah, I actually have a PDF where we tested it and show how it works.", "start": 9158.749, "speaker": "SPEAKER_05"}, {"end": 9170.051, "text": "But let me see if I have a little more detail about the actual interaction design of it here.", "start": 9163.31, "speaker": "SPEAKER_05"}, {"end": 9171.111, "text": "Wrong deck.", "start": 9170.051, "speaker": "SPEAKER_05"}, {"end": 9172.072, "text": "Let's see.", "start": 9171.111, "speaker": "SPEAKER_05"}, {"end": 9174.652, "text": "I want this one.", "start": 9172.072, "speaker": "SPEAKER_05"}, {"end": 9179.153, "text": "No, I want this one.", "start": 9174.652, "speaker": "SPEAKER_05"}, {"end": 9179.393, "text": "Yeah.", "start": 9179.153, "speaker": "SPEAKER_05"}, {"end": 9191.094, "text": "So here we go.", "start": 9185.05, "speaker": "SPEAKER_06"}, {"end": 9193.936, "text": "I got a video of it, so this should help you.", "start": 9191.094, "speaker": "SPEAKER_05"}, {"end": 9197.378, "text": "So this is an implementation of it on the web.", "start": 9193.936, "speaker": "SPEAKER_05"}, {"end": 9203.342, "text": "And so effectively what we do is we bring up, when you tap into the field, we bring up a numerical dial pad.", "start": 9197.378, "speaker": "SPEAKER_05"}, {"end": 9204.503, "text": "So it's all just numbers.", "start": 9203.342, "speaker": "SPEAKER_05"}, {"end": 9205.964, "text": "You can't type anything but numbers.", "start": 9204.503, "speaker": "SPEAKER_05"}, {"end": 9214.589, "text": "And as you type in, you'll note if you enter an incorrect credit card number, it'll throw an arrow and not allow you to continue.", "start": 9206.884, "speaker": "SPEAKER_05"}, {"end": 9221.433, "text": "If you put a appropriate credit card, it'll identify that credit card, switch it to the type you're entering, so it lets you know it's a MasterCard.", "start": 9214.589, "speaker": "SPEAKER_05"}, {"end": 9224.535, "text": "You don't have to set credit card type.", "start": 9222.393, "speaker": "SPEAKER_05"}, {"end": 9228.279, "text": "Then once you get to a valid credit card number, we just slide it over.", "start": 9224.535, "speaker": "SPEAKER_05"}, {"end": 9232.042, "text": "And then you're on to the next field, where you're entering the expiration date.", "start": 9228.279, "speaker": "SPEAKER_05"}, {"end": 9233.924, "text": "And again, it's just numbers.", "start": 9232.042, "speaker": "SPEAKER_05"}, {"end": 9236.046, "text": "Now we show you where your CVV is.", "start": 9233.924, "speaker": "SPEAKER_05"}, {"end": 9238.008, "text": "Again, it's just numbers.", "start": 9236.046, "speaker": "SPEAKER_05"}, {"end": 9242.513, "text": "And this specific example also includes a zip code.", "start": 9238.008, "speaker": "SPEAKER_05"}, {"end": 9244.054, "text": "So if you want to capture that, you can.", "start": 9242.513, "speaker": "SPEAKER_05"}, {"end": 9244.615, "text": "You don't have to.", "start": 9244.054, "speaker": "SPEAKER_05"}, {"end": 9249.916, "text": "But all of this information just gets collected right here on these numbers.", "start": 9245.255, "speaker": "SPEAKER_05"}, {"end": 9253.537, "text": "And you get this real-time feedback, what have you.", "start": 9249.916, "speaker": "SPEAKER_05"}, {"end": 9262.28, "text": "When we tested this with people, to give you some of the detail of what we found, number one, people were very happy to have this big numeric keypad.", "start": 9253.537, "speaker": "SPEAKER_05"}, {"end": 9268.541, "text": "They responded really well to this because they didn't have to jump between keyboards and figure out what's going on.", "start": 9262.28, "speaker": "SPEAKER_05"}, {"end": 9274.413, "text": "They wondered sometimes how to go back to editing their credit card number.", "start": 9268.541, "speaker": "SPEAKER_05"}, {"end": 9276.714, "text": "Not that they needed to, but they brought it up.", "start": 9274.413, "speaker": "SPEAKER_05"}, {"end": 9281.517, "text": "And so effectively, the ways to do that is to either tap on the number or to tap back.", "start": 9276.714, "speaker": "SPEAKER_05"}, {"end": 9284.498, "text": "And both of those things work, so yay.", "start": 9281.517, "speaker": "SPEAKER_05"}, {"end": 9296.725, "text": "And the one thing that they were actually pretty discontent about, we tested a couple implementations of this, including the one Square uses, where they put dots on there, and people hated the dots.", "start": 9284.498, "speaker": "SPEAKER_05"}, {"end": 9298.106, "text": "Because it masked what they entered.", "start": 9296.725, "speaker": "SPEAKER_05"}, {"end": 9300.227, "text": "Does that answer?", "start": 9298.106, "speaker": "SPEAKER_05"}, {"end": 9311.299, "text": "Yeah, so maybe it feels a little complicated walking through it this way, but it's just like if you look at your credit card.", "start": 9304.133, "speaker": "SPEAKER_05"}, {"end": 9312.64, "text": "I think I have a picture.", "start": 9311.299, "speaker": "SPEAKER_05"}, {"end": 9321.346, "text": "If you look at a credit card, you're literally just da-da-da-da-da, da-da-da-da-da, flip it over, da-da-da-da-da.", "start": 9312.64, "speaker": "SPEAKER_05"}, {"end": 9327.131, "text": "And it's all just numbers, and you stay on that input the whole time and put it in there.", "start": 9321.346, "speaker": "SPEAKER_05"}, {"end": 9329.713, "text": "OK, we're to.", "start": 9327.131, "speaker": "SPEAKER_05"}, {"end": 9330.673, "text": "This man's been patient.", "start": 9329.713, "speaker": "SPEAKER_05"}, {"end": 9333.876, "text": "Thank you.", "start": 9330.673, "speaker": "SPEAKER_03"}, {"end": 9343.17, "text": "I would like to talk about ethical way or like philosophical way how we design actually.", "start": 9336.401, "speaker": "SPEAKER_03"}, {"end": 9354.777, "text": "Like I remember from the last conversion of your presentation last year that like spending time on mobile was increasing.", "start": 9343.17, "speaker": "SPEAKER_03"}, {"end": 9362.56, "text": "And then today, you also mentioned that sessions in mobile also getting more often and frequently in the daytime.", "start": 9354.777, "speaker": "SPEAKER_03"}, {"end": 9374.804, "text": "And to build more habit-forming products in the industry now, we are working with people from different fields, like cognitive science, psychologists, sociologists, and so on.", "start": 9362.56, "speaker": "SPEAKER_03"}, {"end": 9382.246, "text": "And what we do as we design is that we capture the user, their micro-movements to get their attention.", "start": 9374.804, "speaker": "SPEAKER_03"}, {"end": 9387.761, "text": "to show, to sell, and so on.", "start": 9383.558, "speaker": "SPEAKER_03"}, {"end": 9398.169, "text": "I just want to ask, do you think building better product necessarily means that we are building a better life for the people, for the users?", "start": 9387.761, "speaker": "SPEAKER_03"}, {"end": 9406.695, "text": "Because they could also use that micro moments time spending, I don't know, having a dinner, or having chat with friends, and so on.", "start": 9398.169, "speaker": "SPEAKER_03"}, {"end": 9410.898, "text": "Do you think necessarily spending more time means a good thing for humans?", "start": 9406.695, "speaker": "SPEAKER_03"}, {"end": 9421.017, "text": "I'm biased, but I personally think it's awesome that interactive devices are taking time away from passive devices.", "start": 9412.23, "speaker": "SPEAKER_05"}, {"end": 9429.203, "text": "Because if you look here, this is in the US, how many hours of TV people spend?", "start": 9421.017, "speaker": "SPEAKER_05"}, {"end": 9433.826, "text": "Four hours a day, on average, staring at a TV.", "start": 9429.203, "speaker": "SPEAKER_05"}, {"end": 9435.868, "text": "Is this good for humanity?", "start": 9433.826, "speaker": "SPEAKER_05"}, {"end": 9440.053, "text": "We're going to have an ethical conversation about that.", "start": 9437.972, "speaker": "SPEAKER_05"}, {"end": 9440.953, "text": "Maybe that's good.", "start": 9440.053, "speaker": "SPEAKER_05"}, {"end": 9442.654, "text": "Maybe that's bad.", "start": 9440.953, "speaker": "SPEAKER_05"}, {"end": 9454.758, "text": "But my small biased opinion, at least as someone who has children, I much strongly prefer if my child is building something in Minecraft than if they are staring at a video.", "start": 9442.654, "speaker": "SPEAKER_05"}, {"end": 9456.338, "text": "That's just sort of a one way conversation.", "start": 9454.758, "speaker": "SPEAKER_05"}, {"end": 9458.399, "text": "And that's just my personal bias.", "start": 9456.338, "speaker": "SPEAKER_05"}, {"end": 9460.66, "text": "Other people may have different opinions.", "start": 9458.399, "speaker": "SPEAKER_05"}, {"end": 9462.26, "text": "So that's one kind of thing.", "start": 9460.66, "speaker": "SPEAKER_05"}, {"end": 9466.442, "text": "And then the other piece is, what kinds of products are we making?", "start": 9462.26, "speaker": "SPEAKER_05"}, {"end": 9475.427, "text": "I think there's a lot of products that are empowering, that give you superpowers, that allow you to do stuff you couldn't ever do before.", "start": 9467.162, "speaker": "SPEAKER_05"}, {"end": 9478.469, "text": "And so I don't view those things as a detriment of attention.", "start": 9475.427, "speaker": "SPEAKER_05"}, {"end": 9482.032, "text": "I view those things as an augmentation of attention.", "start": 9478.469, "speaker": "SPEAKER_05"}, {"end": 9485.394, "text": "By using this, you can now navigate the entire planet.", "start": 9482.032, "speaker": "SPEAKER_05"}, {"end": 9486.995, "text": "You can learn a foreign language.", "start": 9485.394, "speaker": "SPEAKER_05"}, {"end": 9490.117, "text": "I mean, there's all sorts of things that help our lives.", "start": 9486.995, "speaker": "SPEAKER_05"}, {"end": 9494.44, "text": "There's certainly things which are distraction for distraction's sake to sell attention.", "start": 9490.117, "speaker": "SPEAKER_05"}, {"end": 9499.101, "text": "That's called advertising and that's existed for a long, long time.", "start": 9495.699, "speaker": "SPEAKER_05"}, {"end": 9502.223, "text": "And it's not unique to mobile, right?", "start": 9499.101, "speaker": "SPEAKER_05"}, {"end": 9506.605, "text": "People's attention is valuable and therefore you can monetize it.", "start": 9502.223, "speaker": "SPEAKER_05"}, {"end": 9511.808, "text": "So I think it boils down to what are you making and what kind of perspective you take on it.", "start": 9506.605, "speaker": "SPEAKER_05"}, {"end": 9520.092, "text": "My personal perspective is this increase in time spent away from these passive forms of screen sucking are a good thing.", "start": 9511.808, "speaker": "SPEAKER_05"}, {"end": 9523.374, "text": "And we hear all the time, people just keep staring at screens.", "start": 9520.092, "speaker": "SPEAKER_05"}, {"end": 9523.934, "text": "Yeah, they do.", "start": 9523.374, "speaker": "SPEAKER_05"}, {"end": 9527.629, "text": "Sure as hell do, right?", "start": 9525.847, "speaker": "SPEAKER_05"}, {"end": 9529.27, "text": "And it's not the one you're talking about.", "start": 9527.629, "speaker": "SPEAKER_05"}, {"end": 9533.314, "text": "It's not the little powered computer that's in your pocket.", "start": 9529.27, "speaker": "SPEAKER_05"}, {"end": 9542.562, "text": "OK, behind you.", "start": 9533.314, "speaker": "SPEAKER_05"}, {"end": 9543.203, "text": "Hello.", "start": 9542.562, "speaker": "SPEAKER_06"}, {"end": 9544.023, "text": "Hi.", "start": 9543.203, "speaker": "SPEAKER_06"}, {"end": 9545.224, "text": "Do you have any examples on?", "start": 9544.023, "speaker": "SPEAKER_06"}, {"end": 9552.679, "text": "what works for mobile around the filter design?", "start": 9546.497, "speaker": "SPEAKER_06"}, {"end": 9553.179, "text": "Around which kind?", "start": 9552.679, "speaker": "SPEAKER_06"}, {"end": 9557.64, "text": "Filtering search results.", "start": 9553.179, "speaker": "SPEAKER_06"}, {"end": 9558.48, "text": "Yeah.", "start": 9557.64, "speaker": "SPEAKER_05"}, {"end": 9559.6, "text": "Oh, you know what?", "start": 9558.48, "speaker": "SPEAKER_05"}, {"end": 9561.601, "text": "I'm probably not going to have it with me.", "start": 9559.6, "speaker": "SPEAKER_05"}, {"end": 9565.162, "text": "But there is a book by a woman named Rachel Hinman.", "start": 9561.601, "speaker": "SPEAKER_05"}, {"end": 9573.252, "text": "where she has, I think it's called the Mobile Frontier, and she has a series of patterns around filtering.", "start": 9566.107, "speaker": "SPEAKER_05"}, {"end": 9578.436, "text": "I used to have a bunch of this stuff in my deck, but let me see if I do.", "start": 9573.252, "speaker": "SPEAKER_05"}, {"end": 9581.158, "text": "For some reason, I feel like I don't have it here.", "start": 9578.436, "speaker": "SPEAKER_05"}, {"end": 9582.819, "text": "I'll look real quick.", "start": 9581.158, "speaker": "SPEAKER_05"}, {"end": 9590.384, "text": "But she has a series of patterns that you can use for filtering behaviors on mobile.", "start": 9582.819, "speaker": "SPEAKER_05"}, {"end": 9591.485, "text": "Looking real fast.", "start": 9590.384, "speaker": "SPEAKER_05"}, {"end": 9595.845, "text": "Yeah, I don't think I have it in here.", "start": 9594.764, "speaker": "SPEAKER_05"}, {"end": 9599.187, "text": "But if you look up Rachel's book, Rachel Hinman, you'll see it.", "start": 9595.845, "speaker": "SPEAKER_05"}, {"end": 9605.732, "text": "She's got a whole chapter on all of those, hopefully useful.", "start": 9599.187, "speaker": "SPEAKER_05"}, {"end": 9609.034, "text": "OK, onward.", "start": 9605.732, "speaker": "SPEAKER_05"}, {"end": 9614.038, "text": "OK, this guy's got a finger pointing at him and his hand up, so win.", "start": 9609.034, "speaker": "SPEAKER_05"}, {"end": 9616.3, "text": "Thank you for the great stuff today.", "start": 9614.038, "speaker": "SPEAKER_04"}, {"end": 9622.024, "text": "I was wondering what technology will replace smartphones, and when will it happen?", "start": 9616.3, "speaker": "SPEAKER_04"}, {"end": 9623.605, "text": "Oh, awesome question.", "start": 9622.024, "speaker": "SPEAKER_05"}, {"end": 9633.946, "text": "So for years, I've gotten this question around, what's next after mobile?", "start": 9624.9, "speaker": "SPEAKER_05"}, {"end": 9635.286, "text": "Is it VR?", "start": 9633.946, "speaker": "SPEAKER_05"}, {"end": 9636.907, "text": "Is it AR?", "start": 9635.286, "speaker": "SPEAKER_05"}, {"end": 9639.629, "text": "Is it wearables?", "start": 9636.907, "speaker": "SPEAKER_05"}, {"end": 9640.189, "text": "What's it going to be?", "start": 9639.629, "speaker": "SPEAKER_05"}, {"end": 9644.512, "text": "And for years, I didn't have an answer to this question.", "start": 9640.189, "speaker": "SPEAKER_05"}, {"end": 9650.336, "text": "And now I feel like my thoughts around this are coalescing a little bit from being around people smarter than me.", "start": 9644.512, "speaker": "SPEAKER_05"}, {"end": 9661.009, "text": "And the things that I think are coalescing are our ability to take huge amounts of data and learn things from them.", "start": 9651.541, "speaker": "SPEAKER_05"}, {"end": 9672.419, "text": "So if you look at some of the recent advances, and I know there's a lot of hype around this, but I've personally had several instances where I have been super impressed by what we've been able to do.", "start": 9661.009, "speaker": "SPEAKER_05"}, {"end": 9673.78, "text": "And I'll try and find one of these things.", "start": 9672.419, "speaker": "SPEAKER_05"}, {"end": 9688.407, "text": "But we, meaning Google, and the hype, meaning things like machine learning, So just very recently, we announced a couple of things that illustrate just how many parts of our day-to-day lives this stuff will impact.", "start": 9673.78, "speaker": "SPEAKER_05"}, {"end": 9693.368, "text": "And I don't think I have a screen for this, sadly, so I'll just illustrate it.", "start": 9688.407, "speaker": "SPEAKER_05"}, {"end": 9700.451, "text": "One thing that we did is, remember those terrible little CAPTCHA boxes that were all over the internet that you proved to be a human?", "start": 9693.368, "speaker": "SPEAKER_05"}, {"end": 9705.032, "text": "We now, through mining huge amounts of data, can just know whether you're a human or a robot.", "start": 9700.451, "speaker": "SPEAKER_05"}, {"end": 9707.593, "text": "CAPTCHA is dead.", "start": 9705.691, "speaker": "SPEAKER_05"}, {"end": 9712.157, "text": "This is the single biggest contribution of machine learning to date, in my opinion.", "start": 9707.593, "speaker": "SPEAKER_05"}, {"end": 9722.528, "text": "The other thing that we just launched, we launched a program or a system called Razor, which what this does is it, on your device, upscales images using an algorithm.", "start": 9712.157, "speaker": "SPEAKER_05"}, {"end": 9729.494, "text": "So we send images that are 75% less bandwidth to your phone, and your phone makes them look awesome.", "start": 9722.528, "speaker": "SPEAKER_05"}, {"end": 9733.256, "text": "through by looking at billions of different images.", "start": 9730.315, "speaker": "SPEAKER_05"}, {"end": 9738.918, "text": "We write the way to upscale them using an algorithm to make them look awesome on your phone.", "start": 9733.256, "speaker": "SPEAKER_05"}, {"end": 9741.279, "text": "So again, think about the impact of that.", "start": 9738.918, "speaker": "SPEAKER_05"}, {"end": 9751.202, "text": "If we can cut down on 75% of bandwidth of images across mobile, across these 5 billion devices, look at the impact that kind of thing has.", "start": 9741.279, "speaker": "SPEAKER_05"}, {"end": 9756.624, "text": "YouTube recently announced they had projections for 400 million viewing hours.", "start": 9751.202, "speaker": "SPEAKER_05"}, {"end": 9760.103, "text": "per day or something like this, and they crossed a billion.", "start": 9757.781, "speaker": "SPEAKER_05"}, {"end": 9768.711, "text": "And the way they did that is, again, through this deep learning of huge data sets where their recommendations just got awesome and they gave you content that you actually cared about.", "start": 9760.103, "speaker": "SPEAKER_05"}, {"end": 9771.113, "text": "And so they more than doubled their goal.", "start": 9768.711, "speaker": "SPEAKER_05"}, {"end": 9777.739, "text": "And so just with those three kinds of examples, you can sort of see how this will affect all kinds of things.", "start": 9771.113, "speaker": "SPEAKER_05"}, {"end": 9783.344, "text": "And the stuff that I'm particularly passionate about, because you can hear me ranting about checkout and sign-in.", "start": 9777.739, "speaker": "SPEAKER_05"}, {"end": 9784.725, "text": "Imagine how I feel about CAPTCHA.", "start": 9783.344, "speaker": "SPEAKER_05"}, {"end": 9787.447, "text": "Hate CAPTCHA, right?", "start": 9785.786, "speaker": "SPEAKER_05"}, {"end": 9788.568, "text": "Got rid of CAPTCHA.", "start": 9787.447, "speaker": "SPEAKER_05"}, {"end": 9790.129, "text": "You don't have to do anything.", "start": 9788.568, "speaker": "SPEAKER_05"}, {"end": 9798.176, "text": "And it's a perfect example of what I was talking about earlier, where we take all that complexity and do all this stuff just so you can have a normal, awesome life.", "start": 9790.129, "speaker": "SPEAKER_05"}, {"end": 9803.3, "text": "So for me, that's a really big trend that is super exciting.", "start": 9798.176, "speaker": "SPEAKER_05"}, {"end": 9811.306, "text": "And if you look at, again, the advances in translation, the advances in image recognition, this stuff is transformational and very exciting.", "start": 9803.3, "speaker": "SPEAKER_05"}, {"end": 9819.697, "text": "And then the other trend that I think is also very exciting is cue the screen.", "start": 9813.812, "speaker": "SPEAKER_05"}, {"end": 9822.779, "text": "We used to just make software for this.", "start": 9819.697, "speaker": "SPEAKER_05"}, {"end": 9826.762, "text": "Now our canvas looks a lot more like this.", "start": 9822.779, "speaker": "SPEAKER_05"}, {"end": 9829.544, "text": "And what kind of implications does that have?", "start": 9826.762, "speaker": "SPEAKER_05"}, {"end": 9832.807, "text": "What sort of changes the notion of what is a personal computer?", "start": 9829.544, "speaker": "SPEAKER_05"}, {"end": 9835.224, "text": "if you will.", "start": 9833.863, "speaker": "SPEAKER_05"}, {"end": 9844.588, "text": "There are things in the home which can collect data, and the things in the home which can collect data can have output on a variety of screens.", "start": 9835.224, "speaker": "SPEAKER_05"}, {"end": 9852.712, "text": "There's things which can be on you that collect data, and that data can have output on a variety of screens.", "start": 9844.588, "speaker": "SPEAKER_05"}, {"end": 9861.56, "text": "And so when you start to combine those things And you start to combine other kinds of data input things, and you look at things that could be public.", "start": 9852.712, "speaker": "SPEAKER_05"}, {"end": 9871.366, "text": "I think when we build this really quickly, I feel like the next definition of a personal computer is actually this mesh of endpoints, if you will.", "start": 9861.56, "speaker": "SPEAKER_05"}, {"end": 9876.068, "text": "So there's things which out there are focused on input, collecting data.", "start": 9871.366, "speaker": "SPEAKER_05"}, {"end": 9878.73, "text": "And then there's things that are out there which are focused on output.", "start": 9876.068, "speaker": "SPEAKER_05"}, {"end": 9881.086, "text": "which are displaying those things.", "start": 9879.605, "speaker": "SPEAKER_05"}, {"end": 9888.851, "text": "And instead of having input and output just in one device only, it's more of an integrated world.", "start": 9881.086, "speaker": "SPEAKER_05"}, {"end": 9897.136, "text": "And your personal computer is a combination of your interactions in the car, things that are collecting data about you, stuff inside your house.", "start": 9888.851, "speaker": "SPEAKER_05"}, {"end": 9899.317, "text": "This world is almost here yet.", "start": 9897.136, "speaker": "SPEAKER_05"}, {"end": 9903.179, "text": "It's just not cohesive yet.", "start": 9899.317, "speaker": "SPEAKER_05"}, {"end": 9918.98, "text": "And the other nice thing about this, again, if we project forward a little bit, is, in an ideal world, you just sort of pick up any screen, and it is kind of like a dumb terminal to whatever data source or service you want.", "start": 9903.179, "speaker": "SPEAKER_05"}, {"end": 9922.465, "text": "Now you can just fluidly switch between screens.", "start": 9920.543, "speaker": "SPEAKER_05"}, {"end": 9923.966, "text": "Oh, let me throw that up on the wall.", "start": 9922.465, "speaker": "SPEAKER_05"}, {"end": 9926.789, "text": "Let me push this to my laptop, what have you.", "start": 9923.966, "speaker": "SPEAKER_05"}, {"end": 9937.84, "text": "I think a lot of these things exist in micro situations, but cumulatively we're not yet at the state where this stuff is fluid, although it feels like we're getting there.", "start": 9926.789, "speaker": "SPEAKER_05"}, {"end": 9941.202, "text": "So if you take this kind of environment.", "start": 9937.84, "speaker": "SPEAKER_05"}, {"end": 9950.629, "text": "And then you take our ability to make sense of all this data that we're collecting and use it to help you with your life and the examples that I showed earlier and the many more that will come.", "start": 9941.202, "speaker": "SPEAKER_05"}, {"end": 9956.153, "text": "It's going to be pretty damn exciting.", "start": 9950.629, "speaker": "SPEAKER_05"}, {"end": 9958.374, "text": "Yeah.", "start": 9956.153, "speaker": "SPEAKER_05"}, {"end": 9959.615, "text": "I think this might be the last one.", "start": 9958.374, "speaker": "SPEAKER_05"}, {"end": 9960.075, "text": "No pressure.", "start": 9959.615, "speaker": "SPEAKER_05"}, {"end": 9962.137, "text": "We've got to get you to lunch.", "start": 9960.075, "speaker": "SPEAKER_05"}, {"end": 9973.353, "text": "You showed us the numbers on Cyber Monday versus the Singles Days in China.", "start": 9964.247, "speaker": "SPEAKER_02"}, {"end": 9980.557, "text": "Do we have a clue what they are doing different to get these numbers on the mobile device preference?", "start": 9973.353, "speaker": "SPEAKER_02"}, {"end": 9986.984, "text": "Yeah, so what's the difference between Cyber Monday in the US and Singles Day in China?", "start": 9980.797, "speaker": "SPEAKER_05"}, {"end": 9989.327, "text": "In China, they're thinking mobile first.", "start": 9986.984, "speaker": "SPEAKER_05"}, {"end": 9992.21, "text": "I mean, I think this is the baseline difference, right?", "start": 9989.327, "speaker": "SPEAKER_05"}, {"end": 9999.938, "text": "They don't have necessarily this wide distributed PC base, nor do they have this mindset of, oh, desktop, right?", "start": 9992.21, "speaker": "SPEAKER_05"}, {"end": 10001.18, "text": "People convert on desktop.", "start": 9999.938, "speaker": "SPEAKER_05"}, {"end": 10004.484, "text": "They're just all in on mobile all the way.", "start": 10001.6, "speaker": "SPEAKER_05"}, {"end": 10007.728, "text": "And so the deals they offer are mobile-centric deals.", "start": 10004.484, "speaker": "SPEAKER_05"}, {"end": 10015.298, "text": "The experience on mobile for buying has been built from day one that mobile payments are easy and frictionless in their app.", "start": 10007.728, "speaker": "SPEAKER_05"}, {"end": 10024.469, "text": "And again, there are examples of services that are At these higher bars, the Walmarts, Targets, the Amazons, they've made investments in mobile.", "start": 10015.298, "speaker": "SPEAKER_05"}, {"end": 10028.153, "text": "They run campaigns on mobile, they optimize for mobile.", "start": 10024.469, "speaker": "SPEAKER_05"}, {"end": 10032.198, "text": "And it's the combination of those things together that I think really boosts that number.", "start": 10028.153, "speaker": "SPEAKER_05"}, {"end": 10036.682, "text": "In particular, Alibaba, I don't know, 110% if they did it this year.", "start": 10032.198, "speaker": "SPEAKER_05"}, {"end": 10040.144, "text": "But in the past, they have run mobile-only promotions.", "start": 10036.682, "speaker": "SPEAKER_05"}, {"end": 10042.625, "text": "So you only get the deals on mobile.", "start": 10040.144, "speaker": "SPEAKER_05"}, {"end": 10050.729, "text": "You have to go to mobile to get it, which is a great way to drive their mobile usage up, get people used to buying on mobile, and the like.", "start": 10042.625, "speaker": "SPEAKER_05"}, {"end": 10061.662, "text": "And while I think, oops, while I think, you know, we can look at these different countries, I'm going to just bring up one thing here.", "start": 10052.94, "speaker": "SPEAKER_05"}, {"end": 10063.702, "text": "Oops, wrong deck.", "start": 10061.662, "speaker": "SPEAKER_05"}, {"end": 10066.683, "text": "Let me switch over to this.", "start": 10063.702, "speaker": "SPEAKER_05"}, {"end": 10075.305, "text": "If we look at what's happening in even the U.S., here's my nice chart.", "start": 10066.683, "speaker": "SPEAKER_05"}, {"end": 10077.145, "text": "Can we show that slide real quick?", "start": 10075.305, "speaker": "SPEAKER_05"}, {"end": 10081.194, "text": "Which screen is it?", "start": 10080.454, "speaker": "SPEAKER_05"}, {"end": 10082.315, "text": "Do you see it?", "start": 10081.194, "speaker": "SPEAKER_05"}, {"end": 10082.955, "text": "No, not that one?", "start": 10082.315, "speaker": "SPEAKER_05"}, {"end": 10085.396, "text": "OK, I'll move it over.", "start": 10082.955, "speaker": "SPEAKER_05"}, {"end": 10089.677, "text": "I can't see my mouse.", "start": 10085.396, "speaker": "SPEAKER_05"}, {"end": 10090.618, "text": "Where is it?", "start": 10089.677, "speaker": "SPEAKER_05"}, {"end": 10092.098, "text": "Oh, now you can see it.", "start": 10090.618, "speaker": "SPEAKER_05"}, {"end": 10093.999, "text": "Sorry, third monitor over.", "start": 10092.098, "speaker": "SPEAKER_05"}, {"end": 10100.261, "text": "If you look even in the United States, the number of mobile-only users is growing.", "start": 10093.999, "speaker": "SPEAKER_05"}, {"end": 10103.923, "text": "So now nearly one in eight people on the internet in the US are mobile.", "start": 10100.261, "speaker": "SPEAKER_05"}, {"end": 10109.402, "text": "only, whereas the percentage much higher in other countries and regions.", "start": 10105, "speaker": "SPEAKER_05"}, {"end": 10116.846, "text": "But even in developed regions, this sort of behavior is popping up.", "start": 10109.402, "speaker": "SPEAKER_05"}, {"end": 10117.986, "text": "OK, cool.", "start": 10116.846, "speaker": "SPEAKER_05"}, {"end": 10119.347, "text": "Hope some of this was useful.", "start": 10117.986, "speaker": "SPEAKER_05"}, {"end": 10120.888, "text": "Thanks for asking great questions.", "start": 10119.347, "speaker": "SPEAKER_05"}, {"end": 10124.89, "text": "If you want to talk more, I'll be here for another couple of minutes.", "start": 10120.888, "speaker": "SPEAKER_05"}, {"end": 10131.653, "text": "I believe we need to wrap up now so you know which buses to take and you can get to your lunch.", "start": 10124.89, "speaker": "SPEAKER_05"}, {"end": 10133.174, "text": "But thank you very much.", "start": 10131.653, "speaker": "SPEAKER_05"}], "transcriptBySpeakers": [{"end": 4418.424120030303, "text": "Thank you. I am here to stimulate you, it turns out. I wish I had known that earlier. I would have brought some props. And actually, I do have one prop, which is audio. So why don't I try to stimulate you with my audio really fast? Tell me if this works or not. A little bit. Was it not loud enough? Can we make it a little louder? OK, let's do it again. Tell me how stimulated this makes you. Ready? Yeah, a little bit. All right. So welcome to the future. As I was preparing to talk to all of you today, I started pulling together a bunch of my thoughts around what's going on with mobile, where are we at, what kinds of things have we learned. And I realized almost halfway through the process that I was going back to a bunch of examples that I had pulled together about six or so years ago. And I was kind of revisiting these examples and saying, well, what has happened? Have we made progress here? And so that original talk was called Mobile to the Future. And I sort of speculated, where will we get with some of these new technologies and capabilities that we have with mobile devices? So when I realized I was revisiting a lot of these same topics, I thought, well, what better title change than mobile in the future? And we'll see if it actually makes sense as we go here. But what I want to do today, we actually have Two parts together. There is part one that we are in right now. And part one is effectively me talking to you about things that I find interesting. We'll talk about the current state of mobile. We'll talk about optimizing for that current state of mobile in areas that matter for conversion. But then perhaps more importantly, I want to try and inspire us for what's next. I want to talk about the future part of things. Where can we go versus where are we? So hopefully that'll give you a sense for how I think about some of these topics, the kinds of things we can discuss. And most importantly, I hope it triggers a bunch of questions. Because part two, we'll get a 30-minute break, but part two will be you can ask me any questions that you like, and hopefully I will be able to answer some of them. Full disclosure, I have a slide deck with 618 slides in it. So hopefully you ask me something in there, otherwise it's going to be a really awkward 90 minutes after the break. OK, so part one. 10 years ago. And I specifically use the 10 years ago moment because this year we hit a milestone. On January 9, 2007, 10 years ago, I would say we had the moment that most people associate with the onset of the modern mobile era, the thing that kicked off the smartphone age. And you're all probably pretty familiar with this event. This was the introduction of the initial version of the iPhone. And I think it makes a lot of sense. I found myself referencing this moment a lot, because it's such a logical timestamp. It's as good a moment as any to say, here's kind of where this mobile thing really kicked into high gear. And by the way, I see poor people sitting on the stairs. There's chairs up here. You can enjoy comfy things for your bosom, as opposed to uncomfy things, if you so choose. Kirk quoted this man a few times, I'll quote him one more time, and just kind of illustrating what was the intent behind the announcement of the iPhone. And what Steve said is he wants to make a product that leapfrogs what's out there, that is way smarter than any mobile device has ever been. And if we kind of start from this point, if we start from 10 years ago, I think we can actually look at some really interesting and hopefully eye-opening charts that illustrate just how far we've been. And I'm going to say up front that I'm of the mindset that we underappreciate how much impact the past 10 years have had on our lives, especially in terms of distribution of technology and capabilities that technology brings. So let me illustrate what I mean. Steve says we want to make this leapfrog product that's way smarter than any device that's ever been out there. So if we go back in time 10 years, which positions us towards the end of 2006. At the end of 2006, there were actually smartphones in the world. This wasn't a totally new invention from thin air. In fact, the yearly total sales of smartphones about 10 years ago was around 68 million globally. Now, that's only about 30% of the number of PCs, personal computers, that were sold that year. It's about 239 million personal computers globally sold in 2006. So smartphones, pretty small chunk of that, like I say, under 30%. And what were these smartphones we talk about? Does anybody remember what these looked like? Let me remind us. It's our old friend, the BlackBerry Pearl, the Motorola Q, the Palm Trail. Anyone have one of these? There was a little while there where I carried a BlackBerry Pearl around and an iPhone, because, you know, I don't know, that's what you used to do in Silicon Valley. I don't know why. This is the real thing, but I think this thing's awesome, so I'm going to carry it anyway. We were all... Anyone remember the Crackberry Prayer? Are you familiar with that? So the Crackberry Prayer is when you're in a meeting and you're... True story. Happened many times. So this is kind of what the smartphones of 2006 looked like. Pretty different from what we have today. Let's look at what happened since. This was the 10-year-ago situation. First things first, I want to take a look at what happened with PCs. I have to say before I show this chart, because I think you're going to be disappointed in this chart, the personal computer is the thing that helped get the internet into the world. It's the thing that built Google. It's the thing that built eBay and Facebook and so many of the services that are now ubiquitous in our lives and give us so much information and help and assistance throughout the day, let us do so much stuff. So I really don't mean to diminish the importance of the PC when I show this little chart, but I'm going to do it anyway. There you go. Doesn't really look that impactful, does it? Effectively, PCs kind of went up for a little while. They peaked around 300-something million global shipments a year. And then since then, they have been progressively falling after this plateau peak. And if you look between 10 years ago and where we're at today, it's only a 30 million a year difference in terms of sales. Not really the greatest growth rates out there, right? So if that hasn't been such a huge success, and yet digital and the internet and everything is such a huge success, well, let's look at the smartphones number. And here, by the way, tablets weren't so much a thing in 2006. They're not that big a piece of this anyway, but I lumped them together to give you a sense of the overall scale. We're now doing about 1.7 billion of these devices per year. And some people, to me, have complained. They're like, oh, it's flattening out. I'm like, dude, there's only 7 billion people on the planet. There's almost 2 billion of these things sold per year. What do you want? Yeah, it's totally ending. Are you kidding? 2 billion. So clearly, we've got a lot of growth in terms of devices when it comes to mobile. And as I'm sure you recognize and understand, this creates a lot of opportunity. Again, if you consider the PC, this thing is what created the digital economy that many of our industries were built off of, right? This line had so much impact on our planet and our lives. How much impact will a line like this have? And this growth of devices, of course, leads to growth of use. So let's go and rewind our clock one more time back to 2006. And this time we're going to look at usage and audience sizes, not at device shipments. Because maybe they're shipping all these devices and we just have filled the world with a lot of expensive doorstops, right? Well, looking at Facebook's monthly active users all the way back in 2006 again, there was about 12 million desktop monthly active users. 10 years ago, or 10 years later, sorry, 10 years later, we're at 120 million desktop monthly active users. And again, you can see a similar sort of rise and fall on the Facebook side. And at this point, I'm sure you know what the other line is going to look like, but I'll show it to you anyway. This is monthly active mobile users. And that has reached about 1.7 billion people. So it's a very similar pattern to what we saw with devices in general, which indicates to you these devices actually get used, unless they're desktops. Sorry. No, I'm kidding. Desktops still get used and are important in different ways. Pure quantity, right, mobile really dwarfs this experience, at least for Facebook's audience base. And some people will say, okay, well, that's cool for Facebook, but I'm not Facebook. Maybe these big established companies can really benefit from this device growth. But what about new people entering the market? And I think the impact of all of this mobile stuff isn't just felt by established big companies. It actually creates amazing opportunity to reach a huge amount of people in a way that you previously couldn't. So let's, again, rewind our clock. And this time we're going to look at the fastest growing app ever up until 2005. Anybody have an idea what that is? What was the fastest growing app ever, roughly, 10, 12 years ago. Okay, somebody over there said... Somebody over here said... Unfortunately, it's neither of those, but I remember those from the .com era. I think I still hold stock in... The fastest growing app in 2005 was our friend Skype. I knew that. I have a six-year-old daughter who, every time we ever say anything, she's like, I was going to say that. Really? Well, why didn't you? Always. She was going to say it, no matter what it was. So 630 days to hit 40 million users. This was huge at the time, right? Anybody remember how popular and excited people were about Skype? Skype changed hands three times between eBay, private equity, Microsoft. It was a really big deal. And again, 630 days to reach 40 million users was a really big accomplishment. Now, more recently, I'm going to show you another app. This is a Super Mario Run on just one platform, only on iOS. They just released it on Android, I think, yesterday. I saw the headline on my way in. It's now on Android. So this release was only on one mobile operating system. And it took four days. to hit 40 million. In fact, this line is BS. I actually tried to give it a curve, but there's no curve. There's just a straight line on this chart. I'm lying to you, statistics. But what a freaking difference, right? 630 days, and 10, 11 years later, four days. And so this shows that even something new doesn't have to be this established. Now granted, Nintendo, Mario, all these things, maybe it's an extreme example, but it just shows you how the velocity is increasing, the type of scale we now have to operate on. Mobile is at the point where we've reached the state of deployment. There's a platform, a user base, devices, capabilities, stuff that allows us to now take advantage of this amazing scale at which technology operates on the planet. I'll talk about that a little bit more in a second. But before I do that, I thought it would just be fun to talk about how long it took things to reach 10 million users. So we talked about Super Mario Run. It's about, I think, less than a day actually before it hit 10 million. How about Pokemon Go? Any estimates? How long did it take Pokemon Go to hit 10 million users? Any guesses? Two hours? OK. More? Again? OK. That's good. It's not really a number, but I appreciate the effort. Pokemon Go took two days. OK. Angry Birds Space. Any guesses? A week? Originally, the numbers I got were 10 days. Then the Angry Birds space guys contacted me and said, no, it was actually three days. So I got to trust them. I assume they know. OK, here's a fun one. Google+, how many days to hit 10 million users? Nobody's going to get this one. Seven? Somebody guesses? 200? 16. OK. Yay. Twitter. Any guesses on Twitter? 20. Just a bit off. 780. Facebook? 300? At this point, you should note that my chart is going down. Just saying. 852 days for Facebook. And so, again, it just sort of shows you how things are accelerating, right? And the capabilities that we have to reach people. A lot of times you hear people talking about gamification of apps and things like this. While there's certainly some bad things that can happen there, I think there's a lot of things that we can learn from the way Games engage people in the way they spread and the way people get into them. Cuz clearly, they're doing pretty well in a mobile world. Anyway, so the number of devices out there, pretty huge. Audience growth and the potential for audience growth, huge. Well, that's all well and good, but can we actually make any money, right? Is anybody making money off of this mobile stuff? And you're going to notice a common theme here. Again, I'm going to rewind the clock back to 2006. And in 2006, PayPal, which I'm sure you're all familiar with, their annual mobile payments volume was under a million dollars. It was like $900,000, $800,000, something like that. That's how much they did in 2006 in total mobile payments. OK, so from this sub-million dollar number, and you'll notice my chart, by the way, is in billions. Let's look at what happened over the past 10 years. The first few years were just in the millions, so they didn't even register on my chart, because last year they did $102 billion in total mobile sales. So again, we see this huge spiking chart up and to the right. $102 billion from under $1 million in just a 10-year period. Pretty dramatic. And you may say, OK, well, good for PayPal, but my conversions on mobile are lower than desktop. Anybody thinking that? But that doesn't mean the opportunity isn't there. It doesn't mean that all of these devices, all of these audiences doesn't create this immense opportunity for revenue that flat out didn't exist there before. So to illustrate, let's compare the biggest online shopping days of the year. And we'll look at the US first. So in the US, our biggest online shopping day of the year is this thing called Cyber Monday. And in 2006, you get the theme now. By the way, I had to just dig for all this old data. The internet doesn't work so well if you look for things that are older than a day. Have you noticed that? So back in 2006, Cyber Monday, which is the biggest online shopping day in the United States, about $610 million. Pretty good. And if you follow the trend line upward, you can see that 10 years later, it was 3.4 billion. So very substantial growth. Clearly, a lot of people buying things online and quite a big deal. Now, when you look at the amount of sales that happened on mobile devices in the United States on the biggest shopping day of the year, so it's when most people are buying, you'll see it was about 30% of sales completed on mobile. Not too shabby when you look at what is 30% of $3.4 billion. It's a lot of money moving through these tiny little screens in our hands. To look at the different side of the coin here, we're going to look at the biggest shopping day of the year in China, which was actually invented in 2009. It's pretty awesome that you can just invent these things. Cyber Monday was also invented. And so Singles Day was also invented as a holiday shopping day, but in China. In fact, the Alibaba company was really behind this. And so when it kicked off in 2009, it was doing pretty well. And in 2016, Singles Day, $17.8 billion within a single shopping day online. And if you now look at how much of that happened on mobile, 84% of it happened on mobile. So not only is China dwarfing the US in terms of cumulative sales, but they clearly got their stuff together on mobile. And I think this illustrates the gap, which you can call this a problem. I call this an opportunity. Clearly. Some parts of this world are able to get to almost 90% mobile sales on the biggest shopping day of the year. Some of us have not gotten there yet. But there is obviously a path to getting there. The issue is mobile commerce just isn't evenly distributed yet. But the potential, I think, is already evenly distributed yet, which makes it a lot easier to take advantage of this. Because if the potential wasn't there, then it would be a much harder problem. And getting from 30% to 84% mobile sales is a lot of opportunity for retailers. Increasingly, in the United States, more and more of them are actually now able to start taking advantage of this because they focused on it. So Walmart this year, which is a very big retailer, as I'm sure you know, they boasted they had 70% mobile traffic. 60% of their Black Friday is another. I don't know why we have these weird names. It's another big shopping day in the US. 60% of their Black Friday orders happened on mobile, which is double the US average. So if you focus on this and actually work on it, you can take advantage of this opportunity. Target, which is another big retailer in the US, said 60% of their Cyber Monday stuff happened on mobile. Again, double that average. So if you can look at the quantity of money moving and you can start to have an impact here, you can actually make a big dent. And I like these examples because they illustrate work pays off. A year or two earlier, this was not the numbers that Walmart and Target had. And now, through a concentrated effort, they have them. And getting these kinds of improvements on mobile is key, because I think the last 10 years have been very, very transformative, as hopefully I'll illustrate for you in a second. But they've really just set us up for what's next. And so in order to do that, let me look at this chart real quick. So this is cumulative PCs now. This isn't yearly sales that we're looking at before. This is cumulative. How many is there in operations on planet Earth, the rock spinning around the sun today? how many cumulative smartphones are out there in operation. And the cumulative smartphone number is approaching 3 billion. It's probably right at 3 billion now, because these cumulative stats I couldn't pull for the end of last year. But it's about 3 billion. So PCs are about $1.2, $1.3 billion or something like that. Mobile phones, these are non-smartphones. If you look at mobile phones, it's approaching around $5 billion. And this number is actually kind of flattening a touch, because what's happening is mobile phones are effectively becoming smartphones. And so you can expect this line to effectively get replaced by this line over time. So the mobile line becomes the smartphone line. Next line, I think, is the most interesting. These are the number of adults on the planet. And we've got about 5.5 billion out of 7-something billion people on the rock. And I'm going to draw a line here at this point that I kept talking about for 2006, because this illustrates what has happened in the past 10 years. And one, the first thing to notice, we're talking about global scale here. We're literally talking about something that touches all the human beings on the planet. And as smartphones turn over mobile phones, and as that line starts to approach the number of adults on the planet, what are we going to do about that? What happens when all adults on our planet are now connected through networked supercomputers in their pockets? Things filled with sensors, things filled with processing power, things connected to the cloud, able to access all kinds of data, collect all kinds of information. I think it really changes the dynamics. And this is why I say we're entering this world of deployment. These trends are pretty set now. It's pretty clear what's going to happen, unless, who knows, right? I can't predict the future. Hopefully, looking at a chart like this, you're convinced of the mobile opportunity. So huge growth in devices equals big audiences equals big potential for revenue. And there's clearly people taking advantage of that revenue opportunity in huge ways. But what have we actually learned over the past 10 years about making things for mobile? How has our understanding of mobile design changed as we've gone up these lines and we've seen these curves grow in this manner? And that's what I really want to spend most of our time talking about. The first part here is just set up to convince you this stuff matters in case you're not already convinced. I hope you're already convinced. But a lot of times, seeing it from this kind of perspective, I think, helps hammer the point home. And even though we've got three hours together, I can't possibly cover everything we've learned in 10 years. So I'm just going to talk about a couple of things that are interesting to me. Sorry. And in particular, what we'll talk about is sort of the current state of optimizing for mobile, what's possible today. But most importantly, I'm going to look at some kind of weird examples or some things on the fringes that you may say, huh, why are we looking at that? But what I want that to do is to inspire us for what's next. I want us to look forward the next 10 years. Because frankly, what happened for a large portion of this past 10 years is we were too caught up in the past. We're still thinking of everything from the desktop perspective, and you still hear this. Even today, people are like, well, my desktop conversions are higher than my mobile conversions. We're just thinking in terms of what happened before, as opposed to looking today at the opportunity, and more importantly, looking in the future at the opportunity. That's how you actually take advantage of what's going to happen, by being prepared for it, by thinking ahead, by looking toward the future, hence kind of the the purpose of this talk. So 10 years ago, when the first iPhone was announced, things were pretty darn simple, right? There was a camera that allowed you to take a picture, and then you could look at that picture. That was it. Seems pretty crappy, right? Oh, right. I didn't take a picture. OK. It doesn't seem very compelling. But over the years, each subsequent version of the iPhone and of iOS added new features and capabilities. So when we started from that first version, we could take a picture, we could view it. Then all of a sudden, we were able to do video. Then we were able to flip the camera around because there's two, so enter selfies. You could turn a flash on and off. There was now grids. You could have HDR. You could take photos with square boxes, right, kind of stemming from the Instagram explosion back when we got to iOS 7. You can sort of note the design change that happened here when we moved to flat design. Anybody remember flat design? No? It used to be the only thing anybody ever talked about in the design space a while ago. I have to show you my favorite flat design image just because, I don't know, I want to. Where did I put my favorite flat design image? OK, well, this will take me half a second. But trust me, it's totally worth it. It's worth waiting for me to wade through these slides and find my flat design image. Oh, it is going to elude me. Oh, come on. All right, one more. And I promise I'll stop and give up. OK, yes, yes. Window. All right, I'm losing you, aren't I? I swear, this joke is worth it. Just believe me. Oh, come on. I wish somebody could invent something that could take the picture that's in my head. and basically surface it on my screen? Is there something like that that exists? I think Google's working on it. OK, one last second. And if I don't find it, I will give up, admit defeat, and you can chastise me. Ah, yes. Beware flat design. Original Hershey's logo, flat design version, piece of poo. That was worth waiting for, right? Please tell me, else I'll feel really sad later. So yeah, iOS 7 did this flat design thing that everybody freaked out about. Here we go. So we added selfies, we added flash, we added HDR, square photos. Of course, it doesn't stop here, right? Continuing down the trend, we now have slow motion video, time lapse video, panoramas, timers, live photos, portrait photos, telephotos, filtering, you name it. So we've gone from this very simple just take a picture and look at it to, wow, look at all the stuff you can do with the camera in your pocket. And when we look at this collectively, I work all night on the animations for you people. I should actually say that. Let me just say that. These animations that you see, all this stuff, all night. That's all I do. So some of you may look at this animation, which did I mention I worked all night on? You may look at it and say, oh, this is a classic case of feature creep, right? I longed for the days that it was easy to use this stuff. Now there's so many things I can't use it. A very common reaction to seeing this, people say that, right? It used to be such a simple thing, now it's gotten so complicated. And looking at this picture, it clearly feels like, holy cow, tons of features have crammed in here. Other people look at this process and they say, ah, this is the perfect example of building a minimum viable product. You basically build the first version, and people like it. OK, cool. Then you start investing more in it. They're happy with the skateboard. They get happy with the scooter. They're happy with the bike, happy with the motorcycle, and happy with the car. They're just happy people all the way around. And the purpose of this MVP sort of mindset is that you don't spend way too much time adding a pile of features onto something that nobody wants anyway. So you start with something, figure out if they like it. Then over time, you build it up. And I'm just kind of curious, having heard arguments on both sides, which is it? Classic case of feature creep? Who says this is feature creep? Wow, just three people? Okay, interesting. Or I guess the rest of you say this is MVP kind of thing at scale? Don't be afraid. Okay, so seven of you, about 93% do not like to raise their hand in little surveys. I'm going to call that the margin of error here for everything that we do. I just think it's an interesting question. People will debate both sides of it and have very strong opinions on both sides. So of course, the camera isn't the only feature of smartphones to have gained popularity over the years. Does anybody know about this thing called messaging? Of course you do. And if we look at the past 10 years of messaging, again, 10 years ago, you were able to send a text message. I know. I know. It just doesn't compare, does it? Looking at the past 10 years of messaging, we see a similar pattern. We go from this very simple thing to now you can, in addition to sending text, you can hit this little button. You can go take a selfie instantly. You can flip it over. You can do recording. You can send anything from your camera roll. That's pretty useful. You can do searches. You can go find animated GIFs. You can record audio. Send whatever you're saying. I mean, you can literally, this is, I had no idea this thing even existed. If you like fart around on here, God knows what you'll find. You can record a video and draw over it with like fireworky actions. All right, so you're recording a video and you can like light your head on fire and send that to somebody. I guess there's a time and place for that. Perhaps now, right? Oh my God, have you seen his animations? Not only that, there's a whole app store for things to add to messaging. There's a whole category of software that you can insert into messaging to do all kinds of things, to send pretty weddings or wedding doodles to each other. I mean, there's a whole wedding category of messaging apps. Same story with how we share. Quite literally, back in 2006, you know how you share one of those photos you took of the camera? You email it. You would email the photo to another human being. That was how you took the picture from your phone and gave it to another human being. And where are we at today? Well, you have this whole share sheet thing, and inside of there, you can mail, message, tweet, save, hang out, PDF, cloud share, Slackify, LinkedInify, Flickrify, and that's not all. Then you can go into the next layer of options, and you can print, and airplay, and assign to a contact, and put in the cloud, and God knows what else. It's almost a dizzying array of options of what you can do with that photo. I should note, Apple tries to manage some of this complexity. So at first, you get this top-level stuff, and then if you go through all this, you can get to more things, and then only if you go even deeper, they try to stagger some of this complexity. But the capabilities are huge. Of course, this introduces discovery versus usage problems, and on and on we can talk about that. But let's move away from Apple for a minute and look at the same iterative process within mobile app design. So this is the eBay Android app roughly 10 years ago. Looks awesome, doesn't it? And here's an iteration, the next iteration of it. You can see that they added some things, daily deals, messages. And next iteration. Again, I've added a couple things. Right about here is where I think they bought Red Laser, that barcode scanning app. Remember when barcode scanning was going to be the huge thing? Of course, they put the little barcode scanning icon up there, and nobody knew what it was, so then they put a red line through it. And of course, nobody knew what it was, so then they put the word scan next to it. And probably nobody used it. So it stayed there for a little while. And then I think after like version five or six, it went away. And then they kind of moved to search. But the way they did search here, nobody could see search. So that was probably a bit of a problem. And in the next version, you can see that they start to realize nobody could use search. So they just hid it outright. Then they made it very prominent. And then I don't even know what they're doing here. Search is gone again. Here they added deal frenzies, which is awesome, I guess. And now search is back. And da, da, da, da, da. That release, I think, was just to tell you that PayPal separated. On and on and on it goes. And da, da, da, da, da, da. All night. So let's look at this progression all at once. Over the past 10 years, what have we been doing? Are we just moving pixels around? Are we making progress? Are we learning things over the past 10 years? Or are we just putting search here, putting search there, putting search here, putting search there? Should we have search? I don't know. Let's try without search. OK. Put it here. Put it there. Put it there. Put it there. I ask this because this reminds me a lot of what happened with graphical user interfaces. Graphical user interfaces have been around a lot longer than mobile, right? Over 30 years. In fact, let's look at the progression of graphical user interfaces for a second. Here's 30 years of Adobe Illustrator designed for the desktop. 30 years! Are we making progress? Is this good? Did we nail it the first time? Nailed it. I'm just going to move icons around for the next 30 years. First time was perfect. Remember 87? That's when we nailed it. Are we doing the same thing on mobile that we did for graphical user interfaces? Is this the path that makes sense? Are we just applying the same processes and results from desktop design to mobile design? Or are we taking all the things, like the iterations of moving the pixels around time and time again that we did on the desktop, are we just now, hey, let's do it on mobile. All right, now we've been doing it for 10 years on mobile. Is this where we should end up? Or is there really an opportunity to do more? And so this is kind of what I want to explore here. And this is why I think the inspiration piece and the looking out there piece that I'm going to get into a little bit is so important. Because me personally, I feel like we can do more. I don't think the same 30 years of moving pixels around is the opportunity that mobile provides, especially at the scale that mobile brings. Sure, we'll have a big impact if we bring some of this stuff to 5 billion people or so. But how can we create new experiences? How can we really take advantage of this opportunity? And I think there's nothing wrong with iteration. I actually am a huge fan of iteration. It makes a ton of sense, as hopefully I'll illustrate going forward here. But it depends on where we're iterating to. This is, to me, a lot of times the missing piece when people just get into this phase of ongoing iteration and testing. Where are you going? What's the vision? What are we trying to achieve? And so to illustrate this, let's look at an example. I want to thank Craig yesterday for bringing this one up. I didn't know he was going to do that, but it's awesome that he did. He outlined some of the things that I looked at over a couple of years around login. And so I kind of want to bring us up to speed around what's happening with sign-in and use it to illustrate this example of what I'm talking about. Now, you're all probably way too familiar with login screens, right? Everybody seen one of these? Some studies say people see 15 of these a day, which I'm sorry, not what I intended when I got into software design. But you may say, well, what's wrong with this? Everybody knows login screen, username, password, right? Everybody does it. Well, it sucks. That's what's wrong with it. So most people forget a password. I don't know how that's possible, but they do. 5% to 10% of sessions require people going through that heinous request password loop because they forget. The number one request to internet help desks, the number one thing that internal companies deal with is password issues. So this thing actually sucks. It has tons of problems, lots of data out there that shows logging in is a pain in the butt. So one of the reasons why logging in is a pain in the butt, because you can't actually see what the heck you're putting in there. It's just dot, dot, dot, dot, dot, dot, dot. In the interest of security, somebody could be looking over your shoulder. Especially on mobile. If there's a guy in a trench coat and a mustache behind you, just go like this. There's your security. Not being able to see what you're typing is actually a huge issue when you consider what people make you do to passwords these days. So brief aside, call it a rant if you must, I am a registered member of this comic book subscription service, which I turn to to relax. Comic books are fun. They're supposed to be fun. And so I get this message one day, as an added security measure, we have strengthened password requirements. If you have not done so, you must create a new password that meets our updated requirements. I just want to read The Hulk, dude, right? And like, seriously, what if somebody reads The Hulk instead of me? Does it really matter? But whatever. I get locked out. I can't read anything. First, it took me a while to figure out what's going on, because the whole app broke. And then finally, I somehow made my way to try and log in, and I saw this message. All right, I guess I got to create a new password. And then come the password rules. Eight characters, one capital letter, one lowercase letter, one number, one special character, bang, hash, dollar, percent, what have you. How much fun is that? Not fun at all. And so we get forced to make these things that no human being could possibly ever remember. And then when we get told to log in, we can't even see the damn thing. So passwords get really complicated. And so it's really nice when apps actually let you see what you're typing in, so you don't make mistakes, so you can get in. And as we saw earlier, those mistakes can be pretty costly to support. So Amazon, in their sign-in screen, introduced this feature that when you type in your password, you can tap that. Oops, let me come back. I guess I didn't spend all night on this one. When you tap this show password thing, you can actually see what your password looks like, and you can untap to look at it. And this isn't just a nice city on behalf of Amazon, it actually adds up to real revenue. It turns out that inside of the shopping cart flow, if somebody tries to log in and they enter a password recovery loop, three-fourths of the time, they will not complete the purchase. So in many ways, any time you're basically sending somebody off to recover a password, you're bleeding money. And this show password pattern, Etsy launched it for a while on their e-commerce site. They showed that this actually decreases failed logins, which therefore helps you actually grow your conversion. So there's actual money behind this. It's not just being nice to human beings to allow them to see your password. It actually reduces failed logins, which helps grow the amount of people that can actually buy stuff. And why does this work? Well, looking at some studies of how people interact with things, the pattern works like this. You start typing some dots, you hit show, then you can see the password, and then you can type the rest of it in. You know, some of the testing that's... By the way, all these things are linked in sources. I'll give you the slides later so you can take a look at them all. But in testing, people, once they start typing the first character, they'll go and unmask the password, and they kind of understand that interaction to go and see stuff. Which is cool. So this is useful to help people log in, yay. But that method kind of requires some effort on the user's part. They actually have to notice that show button, they have to tap on the show icon to get the benefit of this feature. So in the next iteration that Amazon went through, they changed sort of the dynamics and now the password's revealed by default. So as you start typing, you see your password and you have the option to hide it. and remove it. You can see what you're typing, yay, and then you can say, please don't show that. And again, this makes sense too, because if you simply show people's passwords by default, you muck with their heads. We have unfortunately trained so many people to think those little dots are secure, which that's all I'm going to say. But they think they are. So you muck with people's expectations when you just show them their password by default. So if you just type your password like this, again, looking at data, most people do not expect to see their password as clear text. And many of them become suspicious. Hmm. So how do you fix this? Well, you do kind of what Amazon did, adding this very simple little control, this show password dialogue here, lets people know what's happening and helps mitigate this issue. This is a pretty small change relative to the way changes go, but oftentimes it's these small changes that really add up. I think Craig made that point at the end of the day yesterday with lots of swear words, but hopefully it got through, which is little things can come together to make a big impact. And in this case, instead of thinking this is an error now, 100% of people notice that checkbox, and now they think, oh, this site is watching out for me. This is an improvement in user experience. It's trying to help me log in accurately. Thank you, website. I enjoy you. Taking this a step further, and note the iteration here. This is still the Amazon login screen. This is like version four of just login. They now have a visible version of the password below this hidden version of the password. Again, playing into a lot of the stuff that we saw. So you can see that your password is still quote unquote secure with the dots, but you can actually see what you're typing that's default on and you can hide it if you so choose. So several iterations later, this is the UI that's out there right now. And I think this is an important thing. I think sign-in screens matter, right? Getting people logged in, authenticated, creates business, you name it. And so iterating on it makes sense. First, you couldn't view your password. Then you could tap to reveal it. Then it was revealed by default, but you could tap to hide it. Now it's revealed below as you type and defaulted on, but you can remove it. So is this problem clearly important enough for someone like Amazon to iterate on a lot? But are they really taking this in a future thinking way? Is this more of the same stuff we saw with the 30 years of Illustrator? Are we just moving pixels around and getting maybe like a 1% boost here, 2% boost there kind of thing? Are we optimizing for a local maximum, or are we really moving towards this future that I've been talking about that mobile enables? And what's the difference, right? What's the difference between iterating and actually taking advantage of the mobile opportunity? Well, following all these design updates, Amazon launched the ability to sign in with just a fingerprint on iOS. They use Touch ID, and you don't type in a password or a username or anything. You just kind of put your thumb on a little button And you're logged in. And I think this is kind of a nice small example of what I'm talking about. What's the vision? Is the vision, let's gradually keep making sign-in better bit by bit by bit over a 10-year period? Or is the vision, no sign-in? Mobile devices have a ton of amazing capabilities. They've got biometric readers for authentication. They have built-in sensors, microphones, cameras, geolocation, digital compasses, accelerometers, you name it. There's a ton of stuff in here in terms of capabilities that we can start to use to really change the game. And so iteration is awesome. I definitively believe this has made the Amazon experience better, probably in measurable ways. But we also need a vision, one that is somewhere far out, something that changes the game, a vision of what the future can be. And ideally, it's kind of rooted in the capabilities of what these things can do. For example, oh, we've got fingerprint sensors. OK, let's just off with a single tap. So as we go about optimizing for today, and there's a lot of things we can do to optimize for today, how do we think about tomorrow? What is this North Star? Where do we want to get to? Taking this a little bit further, so Uber, which is a service you probably all know, some of you may like it, some of you may not like it. But they have an online registration form to create an account, make a profile, provide payment information. And it's pretty typical. You make an account and a password. You enter a profile, phone information, enter your payment info, submit to their terms of service, as you do. on the internet, and it's pretty well designed. They've taken a bunch of best practices into account here. So they have this typical sign-up register form, but they also have a complete registration and payment flow using just Touch ID, as we saw on Amazon a second ago. And here, what you do is just one tap on that button, and you immediately create an account and pay for your ride, and that's it. You're done. There's no sign-in form. There's no check-in form. There's no payment info. There's nothing. You just go like this. You're now a customer, and you've paid for the cab that's coming to get you. And when we look at this, what do you think the future is? And this is why I sort of push back against questions like, our desktop conversion is better than our mobile conversion. It's because you're taking desktop forms and shoving them onto little mobile screens. Speaking of payments, which are this key part of conversion, let's take a step back and look at that, exactly what I was just talking about. This is the process of collecting payment info. 10 years later, 10 years after this onset of all these devices and all the learnings and impact we've had, our framing of payments is still rooted in desktop checkout forms, this kind of stuff. Now granted, simply taking a desktop checkout form and making it work for a variety of different devices actually pays dividends. So when Walmart in Canada created a responsive design for their e-commerce site where it would now work on small, medium, and medium-large and medium-small type devices, they saw a big impact. They got increases in conversion, especially on mobile. And I think Ani talked about this yesterday for Alibaba. When they move to a responsive design, good things happen. So just making something accessible on a variety of devices can help use, because you can now use it on those devices. That's a good thing. However, the entirety of what worked on desktop isn't necessarily the best fit for mobile. So when we look at Dell's checkout experience, you can see there's the shipping section, the payment section, the verify section, the order submitted section, all this stuff. Taking all of these things and putting them on mobile, all of it on mobile, and I really mean all of it, is not necessarily the best experience. I mean, I love the fact that they're still collecting fax numbers here. There it is. Daytime fax number with an extension. Of course, that makes sense. And once I've filled in all this information, I absolutely want to not agree to your terms of service. And on and on and on, and so it goes. Oh, my conversion on mobile sucks. Duh. I'm surprised anybody ever filled it out. You should be counting your blessings that a human being made it through this flow. God bless that human. So there's a few, perhaps, obvious reasons this is suboptimal. For one, perception. If something seems onerous, I think the conversation yesterday around cognitive load illustrates this. So this looks freaking hard. Ugh, I don't want to do that. That's terrible. So processes that seem onerous to people they'll avoid, because these are onerous. When people see this, they're going to choose the easiest path forward, which is abandonment, going away. Whereas something like this, which actually captures a lot of the same information, feels, oh, I can do that. I can make my way through this. So that's kind of one thing, taking the time to create this more focused, obvious form can go a long way to addressing the perception issue. But there's other issues. In addition to our instinctive feelings of complexity, you can actually measure the benefits. So those two examples that I showed earlier, I've been showing them here and there. And somebody went and tested them against each other. So they took that kind of multi-dropdown form for flight travel to the simplified version. And they found that completion times Huge improvement. People were able to make it through this thing much faster and easier than they were in the last one, which pays off. So in addition to perception, there's sort of these measurable benefits of improved interactions, completion times yay, and of course, the one we all care about, conversion rate impact. And for a while, I've been showing this Boingo redesign form because I like it for a couple of reasons. A, they sent me a redesign which had a real nice impact on their overall conversion. When they dropped 11 input fields, they got a 34% boost. And again, same sort of decrease in effort that we saw before. But I like it more because I actually had a design that takes it down to four inputs that I thought would be a bit more effective. And now, what's really awesome is a couple of sites have gone and implemented things that look a lot like this. And I actually have real-world examples from very large e-commerce retailers. This is Staples in the United States. And they basically took a similar 22-input screen and got it down to just five. from 22 things straight down to five. And so before when I used to show this, I'm like, I can do it in four. And people were like, whatever. Now I can say, look, they did it in five. It's real. This is not a designer dream kind of thing. And so I think it's worth looking at this in depth. Again, as part of similar to what we did with login, what are the optimizations you can do today to work for these kinds of devices in these situations? So let's do that. Step one here is in this checkout. And by the way, this is guest checkout. If you don't have an explicit guest checkout, you're probably missing out. A guest checkout has been shown to boost the amount of people that can use things by somewhere 30%, 40%. Because in most cases, people don't want to get married to a service. They just want to get in and out. So step one is capturing the customer's name. And you'll notice that there is just a single input field. for the customer's name. And there's a couple reasons behind this, so let me dig into this. And we'll walk through each of these five fields and talk about what's happening. But basically, the principle here is avoiding splitting input fields. And once people are in typing mode, just let them stay in input mode. Don't make them jump between fields in input mode too much. There's a couple reasons why we should avoid splitting input fields. And Baymert Institute has a real nice set of reasons around this. which I'll kind of walk through here. So people have a hard time moving between fields. Sometimes those divisions don't make sense. And sometimes it's unclear what is or isn't required. So we'll start with the bottom one here. When we look at what is or isn't required, of course, you know the asterisks, right? Everybody knows the asterisks. We can talk about that later. But what's required in zip code? Both of these or one of these? Makes you think, doesn't it? You don't know, do you? You can guess, can't you? But you don't know. And this is an example of that issue that we talked about here. It's unclear what's required. So when you split input fields and you have this situation where certain things are required, certain things are optional, that split field doesn't really help people know what they have to do. And if it is required, it can lead to errors. If it's not required, you get the idea, right? So this is one of the problems with splitting input fields. The other problem with splitting input fields, people have a hard time moving between fields. So this person switches. Let's look at this from the beginning here. The person switches to the numeric keypad. OK, great. Now they type in three digits of their phone number. And now it auto tabs, so the keyboard goes down. So now I have to tap the field again, and they have to switch it over to the number field again. Now they're back in the number field. They can type the next three digits, and then, of course, it auto-taps, so it goes down, and they have to go back to open the keyboard, switch over to the number field, tap in the number, and then, finally, they get through that field. What a pain in the ass, right? And so taking your desktop forms that work on mobile and putting them on these devices makes no sense because it is a phone. We can just capture phone numbers like this. And if you really need secondary info, there's ways to capture it, what have you. But the same principle applies to first name, last name. If you've ever observed a usability test, you've probably seen somebody do this. If you've ever watched somebody fill in a form, I've seen this many, many times. People will see a name field, and they'll just enter their first name and their last name, because that's their name. You just start typing their name. You've probably done this, right? You go in there, you say, oh, crap. Oh, I've got to separate name. That's because your name to you is a single coherent entity. And this is the example of that earlier comment from Behmer. People find division illogical. meaning this is my name, I will enter my name, I don't think of my name in necessarily these parts until you tell me to. And so a single name is also more representative of global differences. There's people who have a single name, there's people who have eight names. True story. All right, so you can cover all these variable differences. The third reason why we're down to a single name is to avoid the next input field, which says name on credit card or name on payment method, which then you have to ask yet again, because now you split the name, so you need the name that shows up on the credit. Here, you don't have to ask it again. You just ask for their name. So we got their name. Yay. What are we going to do about shipping address? We were able to collect a name. That's good. For here, what we're actually going to use is a principle called inline validation, which is helping you as you type. It's real-time feedback that helps you avoid errors, helps you get to correct answers. And so what Staples has done in this case is made use of the Google Maps autocomplete address API. And the really neat thing about this API is you can restrict it down to particular countries or not. Basically, you start typing, a dropdown with a bunch of autocomplete suggestion comes up, and in pretty much all cases I've ever seen, the address is there before you finish typing it. So you save a bunch of time, you improve accuracy, you minimize errors, and just by typing 712 second, you hit that button, gone. Uses geocation, helps you manage all this good stuff. So here, through inline validation, we've now collected a shipping address, and we've done our best to make sure it is a valid shipping address. And we haven't had people jump between 20 different fields to do so. Saving effort like this pays off. We did some tests not that long ago, comparing collecting city, state, and zip. Now, granted, this is in US addresses, so different regions have different variants of what you can look up and how you can look it up in addresses. and different structures. Again, we can talk about that more deeply in Q&A if you want. But here we were collecting city, and then collecting state, and then having people enter their zip code. Whereas in this version, you entered your zip code, and we would automatically go and fetch the city and state based off of the zip code that you put in. which was cool, so we infer that automatically. And when we compared the two experiences, we found doing those three fields, city, state, and zip, took about 21 seconds, and this took about eight seconds. And looking at personal preference, the majority of people strongly preferred doing it this way as opposed to this way, because it saves them a bunch of time and effort. So taking this sort of principle and applying it to the whole address field helps a lot as well. Now, we can use a similar technique for capturing email or contact information. And so again, we'll use inline validation here. And as you start typing your email address, we'll do an autocomplete based off of common email addresses. And you can, again, tap on the thing that is right for you. And you're off to the races. Why do we have this? Has anybody seen this repeat email address thing out there? You've probably encountered this. Why the hell do people do this to us? It's cruel. It's unusual. It's punishment. The reason why they do that is to get rid of this thing called hard bounces. So on average, 15% of times you collect an email address, it will hard bounce, meaning it's not a valid email address, which is not what you want when you're setting up a new customer relationship. So you ask twice to make sure that errors get cut down, what have you. Our friend inline validation can get rid of this really erroneous stuff. So when you type something that looks kind of like a Gmail account, but they mistyped it, you say, did you mean that? You add it here. This sort of interaction will cut down on hard bounces by a huge amount. You can never get rid of the people who type joe at fart.com because they hate you. Actually, what they hate you for is making them register again. I had a real funny story for half a second. I did this two days ago. I had to register to download a white paper. And I put in bob at fu.com. And they're like, that email address has already been taken. True story. So yeah, verifying email address with a real-time message will get rid of some of those pesky balances. Cool. So now we've got name, we've got address, we've got email. Last but not least, we want to collect payment info. And there's a couple things happening here. First, I'll show you the input mask pattern. And then I'll just sort of reference, they also enabled the ability to scan a credit card. So this is kind of available now, mobile, where you can just take a picture of your credit card. I have no stats on how much people actually do or don't use this. But it's an example of an attempt to try and use the sensors on the device to make this process easier. You just point your phone at a credit card, it collects all the info, and you're done. Pretty neat. But let's look at the input mask implementation. And the way this works is, note you're up with a big dial pad, so you're just collecting a bunch of numbers, you start typing your credit card number in, it highlights the type of credit card you have, because there's a pattern behind these, you don't have to select the type of credit card you have, so we can get rid of that input field. We highlight that, and then what we're going to do is scoot this thing over, and now since you still have the numeric key, you just type in your expiration date, and then based on the credit card type you have, we change the icon to show you where your CVV security code is. You type that, and without ever leaving the dial pad, you've entered all the information you need to process a credit card. We've done a bunch of testing on this, and this format is actually very successful. We tried it out on Android and iOS, and it works. So last but not least, we've got name, address, contact info, credit card. And then we have a smart default set here, which is use this address as a billing address. You can undo that and change it if you want, but in many cases, you don't. Here's how much it's going to cost, shipping, tax, everything you need. Hit the button. You're done. All right, five input fields, done. And again, we see a lot of iteration on what is a very important issue for a retailer. Check out, huge important issue for a retailer. We saw them reducing inputs, doing address lookup, trying inline validation, input masks, scanning credit cards, using smart defaults, all sorts of techniques. And the impact of this was actually pretty dramatic. They had a 50% reduction in checkout time, 1,000 basis point improvement in people who started checkout, engagement went up 40%, 70% reduction in clicks, all sorts of good stuff. And there's a lot of big impact from this. But again, I ask the same question. Are we still just sort of moving pixels around? Are we optimizing for a local maximum? Or are we really moving towards the future that mobile enables? And again, I'll say the same thing. I think there is a lot of value to doing this. If you go do this for your checkout flows, probably good things will happen to you. A lot of improvements to be made. And hopefully just showing you these kinds of improvements gets ideas going in your head. Oh, we could do this. We could do that. Why don't we try this and try it? Great. All sorts of optimizations to do and clearly examples out there of those kinds of optimizations moving the needle in the right way. Going from 22 input field form down to a five field input form that reduces errors dramatically, that seems like a win to me. But let's go back to Amazon for a second and check out how their payment process works on mobile. So here I see a lovely little live pet that I would like. And so I will hit buy now. And then the only thing I do is I just swipe. Done. It's going to be at my house tomorrow. That's pretty awesome, right? I just hit buy now, and then I just go swipe, and the little lovely pet is at my house tomorrow. Super fast process. Again, to echo some of the talk around psychology that I think kicked off the conference yesterday, reducing effort, reducing cognitive load, making it frictionless, fast, and easy for people to buy things. There's a reason Amazon is doing so well, because when you see something, you just got to go, and you paid for it. Now, granted, they had to collect this upfront information first, but maybe you could use that pattern I just showed you a second ago from Staples to do that one time. So that's pretty painless to get at one time. And then after that, you start thinking more like this. And not only is this process super fast, but it actually aligns with how people actually use mobile devices, which is specifically one-handed. There's a lot of one-handed use of devices. And as devices become bigger, as our phones continue to become more fablity, things that are up high become more ouchity to use. So this is sort of the comfortable, nice-to-hit area on your phone. And so these interactions that are down in the bottom, like that swipe and tap, really works a long way. Our fingers are only so long. And when we start to build things that are designed to be used in our hands, we have to think about how far can we, especially one-handed, reach our fingers. They only stretch so far. And looking at this in terms of market trends, you can see that these big phones, these phablet-sized things, five to seven inches, are really increasing in share at the expense of these smaller-sized things. So when we work on designs, we really want to make sure that we're thinking about these market trends, hoping that there's some longevity behind what we build, so we don't just, again, build for today, but we look at where the market is going. Ani showed this payment request API that Google has been working on with web browsers. And I really love the philosophy of this, because it illustrates kind of what I was talking about a second ago, which is you come over to, say, a merchant on Shopify. You hit the Checkout button, upslides this little thing that has all of this information, because it was stored in the browser. And you just hit Pay, and you're done. All right, super frictionless, super fast, super easy. Granted, browsers have to implement this. And granted, people have to go and put their money credentials and off inside the browser. But if it's there, wow, it's almost as fast and easy as that Amazon system by taking your pre-populated info and giving you this single action pop-up. So once again, we see Amazon not sitting still in their implementation of this and continuing to iterate. We saw a lot of iterations on the payment experience. But once more, I want to ask, what are we going toward? What is the bar here? What is the North Star? Where do we want to go with all this iteration? One tap and a swipe, that's pretty damn good. Can we go further? Or can we take this? And certainly, Amazon thinks we can go a lot further. Let's look at some of the stuff that they have done recently. This is the Amazon Dash button. This is a little thing that has Wi-Fi, it's connected to your account, and it's effectively a single-use device. It only does one thing. You know what this thing does? Orders Tide for you. So you tap it one time, laundry detergent shows up. That's it. And in context, it's actually pretty darn useful. You put it on the washing machine. Where do you run out of washing machine detergent? In the freaking laundry room, right? You don't think, oh yeah, I need more laundry. Maybe it pops into your brain somewhere, but right here and there at that moment is when you need the stuff. So you tap it once and you're done. There's no browsing, no opening an app, no finding the app, no clicking over to search. I mean, think about all the steps this removes, right? You just go like this. OK, it's coming. Versus, oh, let me unlock my phone. Let me hit the app. Let me do a search for Tide. Oh, is that the Tide I like? I don't know. Is this the Tide I like? Oh, OK. Now let me do it. And then I get my buy and swipe. But this eliminates all of that. Press it once, and detergent is on its way. In one tap, in context. And this is what I promised to be one of those weird examples, because you're like, I can't do that for my insurance thing. Maybe not. I know the insurance question is going to come up later. How can I do this with insurance? Thank you. You have too much money already in insurance. You don't need to do anything. Just keep doing what you're doing. You have all the money you could ever want. Don't be greedy. I'm serious. Insurance is a lot of money. But I don't mean necessarily go and implement an Amazon Dash button for your thing. But have this kind of vision in your head. How far can we take it? What is the North Star we should go for? Let's look at that in another example. Amazon has this new line of physical stores called Amazon Go. They boast no lines, no checkout. To me, this is an example of an awesome North Star. Instead of let's optimize checkout, how about no checkout? I love that. Our goal is to have no checkout. And if you have that kind of vision in your head, no checkout, then you can see how you might start behaving differently. You could still go through the iteration process, but as you're iterating, you're constantly trying to get to this idealized end state, something that goes beyond what you have today, something that feels like the future or is the future. So let's look at this for half a second. Here's how Amazon works from a user perspective. And I apologize for the marketing-y music thing. I'll come back and talk after it.", "start": 2.532, "speaker": "SPEAKER_05"}, {"end": 4449.225588761905, "text": "Four years ago, we started to wonder, what would shopping look like if you could walk into a store, grab what you want, and just go? Thief! What if we could weave the most advanced machine learning, computer vision, and AI into the very fabric of a store, so you never have to wait in line? No lines, no checkouts, no registers. Welcome to Amazon Go.", "start": 4423.509, "speaker": "SPEAKER_00"}, {"end": 5957.0957788823525, "text": "OK, so how does it work? You just walk in, you tap your phone on the thing, you take what you want, and you leave. They call it walk-out shopping, right? And it's an attempt to create a frictionless shopping experience. Now what's really going on behind the scenes, because that's a whole other story, machine learning, facial recognition, microphones. Let's look at this for half a second. So how does Amazon Go actually work? So when you come in, you have a little app on your phone that you scan with the sensor. It identifies that user. There's a camera. And the camera will actually use facial recognition to uniquely identify each shopper. And then throughout the store, this is the creepy part, right? Cameras see what you do, and whether that item stays with you or not. There's images captured in possession of the item, we just talked about that. There's microphones that know where you are and where people are, so they know where to kind of point the cameras. There's infrared pressure and load sensors on all the different shelves. Note when an item is picked up or picked back. And then there's a whole bunch of kind of deep learning and or machine learning kind of systems that make sense of all this data behind the scenes to tell where people are, because literally what they're doing is they're tracking everything in the store in real time, including human beings. So they have a map of, okay, this person's over here, these items are in flight, these items are with this person, this person left with these items, what have you. A lot of complex stuff, right? Facial recognition, microphones, cameras, machine learning, all this stuff behind the scenes. And the interesting thing about this, in my opinion, is all this complicated stuff This technology is very complicated. Think about all the things that they're doing in their track, and all of that stuff is going on from a hardware and from a software perspective to make a very simple and frictionless shopping experience. And this is the trick in a lot of what we do in the technology industry, is take all this technical complexity, bring it on yourself. Take it on as a burden, and you take it. Don't put it on the human being. You deal with it. All the human being have to do is walk, take thing, and leave. That's how you make a happy human. You take all the benefits that they get from all that technology, and you throw it behind the scenes. They don't even have to think about it. They just go and take things. They feel like they're stealing. They're elated. They're kind of nervous. Oh, my god. Whereas the machines do all the heavy lifting. Again, why I like that example of Amazon Go is because I think it's such an illustrative example of this. Look at all the stuff Amazon is doing to allow you just to walk in and walk out of a store. No lines, no checkout, no anything. Coming back to the Dash button, this is another great example of that philosophy. This thing has Wi-Fi, it connects to your account. That account is built onto a complicated billing system. That billing system has returns, that billing system has processings associated with it. It's even connected to a much deeper distribution and delivery network where there's warehouses around the entire world that do the distribution and they work with carriers. I mean, there's a gazillion things happening behind this tiny little button when you push it. What do you need to do? Push the little button. Everything else behind there is then working for you. And it ultimately delivers this very frictionless and natural experience. And again, when we want to be inspired by things, to me, this is kind of a nice North Star-y thing to dwell on. To stick with this theme for a second, let's come back to Apple for a minute. I think the new AirPods are a great example of this kind of moment as well. So in order to connect your AirPods to any device, you just open the lid. That's it. You open the lid, a little thing pops up. It says connecting, and they're connected. You're done. I didn't touch any button. I didn't open any app. I'm not hitting any UI, nothing. I just flip open the lid on the case, and I connect it. Now, you want to start playing music, what do you do? You just put it in your ear. It starts playing music. If you want the music to stop, you just take it out of the ear. I know, you're all stunned. Like, no, it can't be like that. I got to hit a button, right? It's got to be an app. I got to sign into an app. Nothing. It's this frictionless, graphical user interface-less kind of natural interaction. And this term natural user interaction, NUI, it's been batted around for a while, but the general concept is we create interfaces that are based on a lifetime of learning. Amazon Go is a great example of a natural user interface. You walk in, you pick something up, you walk out. You know how to do that. You've been doing that your whole life. Right? So here, you want music to start playing, you just put in, done. Now, if you want to charge these things, you just put them in the case. Case charges them. Pretty darn cool, right? Another example of a natural user interface we saw at this conference. Google Home, just talk to it. It'll give you answers to things, help you manage your text. You know how to talk. Most of us know how to talk. Some of us have some problems talking. But we know how to talk. So see a pattern here? between the Dash and the AirPods and the Google Home. I think the pattern here is this differentiation between these natural user interface type experiences that we know how to use based on a lifetime of living. They're frictionless, and I think these things serve as a really great north star for where we can go. Can we design checkout with the goal of no checkout? Can we design sign-in with the goal of no sign-in? It's these kind of aspirations where I think we can do more than just incremental improvements. Because while, again, I'm going to jump back to this picture that I showed earlier around sign-in. This is what I'm really interested in. What's that moment where we just change the situation? Where do you go from this ongoing iteration, which I'll say again, has value. I'm not poo-pooing this, right? There's a lot of value in this. But what's the moment where the bit flips and the dynamics change? And all of a sudden, registration is just putting your finger on something. Or all of a sudden, buying something is just walking out of a store. I think those moments are really, really interesting. And just being inspired by those goes a really long way, hence why And I keep showing these natural user interfaces because I think they're a great North Star from an inspiration perspective. And coming back to these for half a second, what better example of a natural user interface than using your eyes to see? And so this new thing from Snapchat, Spectacles, make use of a camera on these lenses to give you 180 degree of what you're seeing. It's actually pretty cool. You can just turn on recording by just tapping the glasses. That's it. But then, This hardware experience is just so simple. You wear glasses, you tap them, it records what you see. Wow, super easy. The software experience is almost the exact opposite. And I had this moment hit me when I was using these things and I tried to get the video off of them and I felt like I went back. in time, 10 years. Because to use these things, to get the video from them and send them to someone, I have to take out my phone. I have to tap on an app. I have to open the app. I have to go and tap on my memories. And in my memories, I have to find the thing I recorded. And then I have to go and flip through these things and tap multiple times to find the video. And then when I find the video, I have to tap on Send. And then after I tap on Send, I have to pick who I'm going to send it to. And then I can actually send it. Remember how I said we want to hide complexity from the user experience? This app open and UI dance is not a great example of that. And many times what we do with our software is we actually show that complexity to people. In the Amazon Go example, we're trying to bury all that complexity, get it out of there. Don't make people worry about it. But what we do with software a lot of times is we shove it in people's faces. And why? Because we can't make a decision. We can't decide what matters most right now. And so what we do is we just throw every option at people. Oh, I can't decide what matters at this point in time. So here you go. Just take it all. This was very easy to do when we were designing desktop screens, because you just throw every big screen, throw everything you want in there. I don't know. What the hell do they want? Horoscope? No clue. They probably want to see this woman dying. I don't know. Maybe how about the Netflix rating system? Sure. Throw that in there. Let's throw it all in there and show everybody everything. You can't do that on mobile because the screen is tiny. And so you have to make these important things called decisions. And mobile, in my mind, is all about getting people the right thing at the right time. And so I'll just show you two simple examples of this. Let's come back to messaging for a second. So I get a note that says, what's your address? And without me doing anything, there's two little buttons that show up right underneath the reply that say, work, home. Please don't visit me. And I just tap home. And I'm done. That's it. This is a great example of the right thing at the right time. And it's just like one tap, just like the Amazon Dash button, the Snapchat recording example. This is why I think those natural user interfaces are good inspiration points. They get us thinking like this. They get us to think, well, what is the thing that right at that moment, at that right time, I think somebody echoed Jeff Bezos earlier, where he said, we start from the customer and iterate outward, right? What is that customer perspective? What is the need that people have, and how can we fulfill it? Sticking with messaging, shifting over to Android, you get a message in a language you can't read? No problem. Just tap it. You get tap to translate right there. And without ever leaving the app, you can translate that text. You can even then reply in that language. It's another example of just a very frictionless, in the moment, right time, right action kind of UI. And why do we care so much about these fast frictionless experiences? Am I setting the bar too high? Will this ever work for insurance? Why do we care about such immediacy? Many of you are probably thinking, well, yeah, OK, that's cool. But things I do can't be like that. My stuff is complicated. It can't ever get simple. And sure, maybe there are some things that can't or shouldn't necessarily get that simple. But it sure can't hurt to try and go there, right? Why do I raise the bar so high for mobile design? Why do I care so much about these fast, frictionless, right time, right action interfaces? Well, let's look at how people actually use their mobile devices. Here we see mobile 3G traffic patterns. So these are different kinds of devices connected to a cellular network. And they're all on the same kind of network. The only difference is there's different kinds of devices. And so you can see laptops, tablets. And what you notice about smartphones is there's a lot of these short, bursty sessions, a lot of in and out on the network. contrast that to these bigger, longer, more sustained sessions on laptops. And if you dig in a little bit further, Apple recently shared how often people log into their iPhone. They said it's 80 times per day on average. An on-device research study of Android found similar numbers, 76 times, and extreme users have higher numbers or so, but these are sort of averages. So there's lots of sessions. roughly like 80 something a day, which matches kind of what we see here, right? Supports this data, lots of small singular sessions. So lots of sessions happening on these devices. And if you look at what these sessions look like, there are certainly some long ones. That is very true. Occasionally you watch Netflix, play Angry Birds or whatever. But most fall, around 55% fall under 30 seconds or less. And then the average is somewhere between like 80 to 120 seconds. So it's not a lot of time per session. Each session is really short. There's a lot of them, but in general, they tend to be short. And so understanding this context helps explain why I care so much about these fast, frictionless experiences. I'm trying to design for the reality of how people use these devices. They use them a lot, but they use them in small doses, quick chunks. And so having this forced concentration on building for that reality helps a ton. I think these numbers also illustrate, remember that talk about mobile performance, In this chart, what do people hate the most when browsing the web on mobile? They hate slow things. Why do they hate slow things? Because they're only on the damn thing for 30 seconds. If it takes 10 seconds to load, no wonder people dislike slow experiences. And this is why solutions like things like skeleton screens, I figured I'd just throw this in here since we talked about performance earlier. So skeleton screens are effectively these little baseline grids that make it look like content is coming in so you get a sense that things are actually happening, but they load very, very quickly because they don't grab images. and actual content yet. Actually, I'll show you how this works. My favorite example of skeleton screens is from Google Photos. So from Google Photos, when you load up a page, you get this grid. It sort of shows you the relative proportion of images. For example, you can see this is a panorama. A little hard to see on the screen here, but it'll identify what's a video and how long it is. So you get the structure of your content coming in instantly. It's already there. And then what we do is we gradually load in the content dynamically. Well, we try to be smart, so we'll fade in things as they come in. And then the other thing that happens is the content actually loads where you're scrolling to, not where you've been, so that it anticipates your movements and is there and ready for you when you show up. So lots of things you can do. And all of this stuff is around perception of speed. There's a certain amount of speed that we can get from loading images and optimizing them and doing all that. But then there's a lot of things that we can do in the user interface to make it feel like something's happening to create the perception of speed. And again, another topic we can discuss in QA if you like. OK. Coming back to why these solutions exist, mobile interactions are frequent but short. Another study that put cameras on people for a couple days and followed how long they looked at screens in their lives found it was about 38 seconds on average. So similar data again, but the reason why I'm referencing the study is because what they were primarily looking at is smartwatches. And they put cameras on people, they watched them throughout the day, and they saw how often they looked at their smartwatches and what they did on them. And you'll find that the bar on these kinds of devices is even more extreme. The average interaction time is about seven seconds. Way more hardcore than mobile phones, right? And I like this because it offers another opportunity to learn. This is my continuing thing here. Let's look at parallel stuff to see what can inspire us and can help us go even deeper on mobile. Because if you are designing things for a 7-second world, you're going to kick some serious butt in the 30-second world. So let's just look at a couple of these smartwatch interactions. In this workout app, they're used to you run a workout, and then it tells you you can save or discard it, and then you have to either hit one of those buttons. Sometimes you hit it on accident. What happens in the Second version is it just automatically saves for you. So a nice example of taking the burden on ourselves instead of putting it on customers. And this is another way you can save a step. You can save some effort. You can save some thinking. Just do things for people when they make sense. You've been working out for 30 minutes or an hour? I'm going to save this so you don't freak out when you lose it later. You can discard it after the fact. Sticking with the same workout app for half a second, The first time you use it, you get this series of choices. You can either do a walk, run, cycle. Second time you use it, what it'll do is learn what you did the first time, and it'll put the last type of activity that you did at the front and make it nice and big. So this is an example of learning from past behavior. You've done something once. We don't need you to repeat it yet again. Here you go. Shifting over to another app, this is a little timer app. And in the previous design, you had to meticulously pick how long you want the timer to run, and then you hit start, and then it finally starts. In the next redesign, what they did is they just made these super fast and simple defaults. Customization is down lower if you need it, but you just tap it and you're up and running. So this is a nice example of setting some custom frictionless defaults. And this last one I think is, or is this the last one? Maybe the second one. This one's my favorite because this actually changed how I use this device. So if you looked at the way they did messaging before, you had to tap to get the option to reply to someone's message. And then you hit reply. And then you get a choice of how to reply. In the new version, right there below the message, you get the little microphone. Super fast and easy to actually respond. Before, when you had to tap multiple times, it was like, oh, I might as well just get my phone. Now, you just go, boom. And instantly, it's faster and easier than actually using your phone. Again, small stuff, small changes, but it's shifted kind of the behavior. And I lied. One more smartwatch example for a second. Again, in this case for inspiration, this is one of the things that I think illustrate to me how natural user interfaces and software and all the capabilities of these devices can come together. Because right now, I can get a light little tap on my wrist. There's a haptic engine that sort of taps me. It goes, bup. tells me something's new. I twist my wrist up, it uses the accelerometer on the device to know it's changed position and tell me, here's what I blipped you about. This satellite is about to pass overhead and you can see it. Since you're an astronomy geek, cool. Right? So I just tap on it once, and it uses the magnimeter, the digital compass, to show me where that satellite is relative to me. And if I want to, I can tap in again, and now it uses GPS to say, here's where you are. There's that street. You should look in that direction. That's pretty freaking cool. It's using all this stuff, the GPS, the magnimeter, the accelerometer, the haptic engine, and all this data and technology behind it. But to me, it just feels like, hey, satellite, where? Oh, right there. If you like to look at satellites. Some of you might not. My kids get a thrill of seeing the ISS fly overhead all the time, so it's pretty neat. Anyway, what can we learn from smartwatch app design, where this time to interaction is super extreme? We looked at ways to minimize effort, learn from past behavior, and use these sensors, all that good stuff. And all of these are really good insights that we can then bring to mobile. And so I'm just going to end with two examples of doing that, and then leave you to break, and we can talk about all this good stuff going forward. So things like pressure-sensitive inputs. If I'm looking at a list of music, and I do this thing where I just push a little harder, Right then and there, I can go add it to a playlist, download it, delete it, remove it. It's this really fast and easy shortcut inspired by a quick little gesture. Of course, I can still tap regularly, go to the page about the song, tap the little meatball menu, and then get those choices. It's certainly there because the other gesture is a little hidden, but this nice, fast, and fluid shortcut helps a lot. Here's another example of these sort of fast and fluid shortcuts. And this is a technique called fluid gestures, fluid touch gestures. So here what I do is I just press and hold on the microphone. It starts recording. I just move up and let go. Audio gets sent. So again, I just press and hold. Starts recording as I talk. I move my finger up. gets sent. One fast, fluid touch gesture that does multiple things at once. And contrast this type of fluid gesture to something like the current UI in Maps. You have to touch Search. You go and say, I want to go home. And then it comes back. And you can say Start. It's this multi-tap, very GUI-like experience for getting things done. And if you contrast this to a fluid touch experience, you just hold down Search. Up pops a list of options. Drag up. Let go. and you're navigating away. Again, more inspiration than anything else, but when you start thinking about these fast, fluid, frictionless interactions, you can see how you can start thinking about making things that work like that. All that aside, hopefully some of these examples from hardware, from smart watches, from natural user interfaces, from fluid touch gestures, one of those things I hope has inspired some of you, because that's the point here. And the point is we need to know the experience we want in the future so that we can work toward it today. Mobile has created this huge opportunity for us, for audience growth, for revenue. All these devices out there are at global scale. Huge area open to us for deployment. And we can, using a bunch of the techniques I showed you today, optimize for today. There's ways to take critical interactions like checkout, critical interactions like sign-in, and make them a lot better. You can go and make these improvements right now. And you can probably end up in a state like Amazon or Target that I showed, Walmart and Target that I showed earlier, where they're double the industry average, right? Instead of 30%, they're at 60% sales on mobile. All this stuff is possible today, provided you put in the effort. But we also need to look past today, toward the future. And if we look to the future, I think there's an opportunity to remove this friction even further, to take all this technical stuff and put it in the service of human beings, so we can just go live our lives. So we can just do what comes naturally to us. And we use our understanding of technology to enable that. And if you have a vision like that, I think it helps with the optimization and iteration process, because you kind of know where you're trying to get, and you can evaluate your steps. The future, I think, is going to be even more seamless if we do our jobs right. The other point I'll make is the next 10 years, high degree of likelihood are going to move even faster than the previous 10. So the sooner we get ready for them, the sooner we start thinking about them, the sooner we start aiming for them, the better off we will be when they arrive. And those are kind of the three big things I want to leave you with. So thanks. So as promised, first part was me blabbing on for a while. Second part is us talking. So we have this microphone, which I will do my best to throw properly. And I am here to answer your questions. Hopefully, there's things that are on your mind that we can talk through that will help you. And who's up first? Either I can't see, or you're all way too shy. OK. All right, here we go. Ready for it? Oh, that's not even close. You have to ask a question. Double-handed. Didn't even work.", "start": 4454.902, "speaker": "SPEAKER_05"}, {"end": 5958.48636, "text": "All right.", "start": 5958.398, "speaker": "SPEAKER_11"}, {"end": 5959.658133333333, "text": "OK.", "start": 5959.338, "speaker": "SPEAKER_05"}, {"end": 5967.022592592592, "text": "Well, you were talking a lot about the user interfaces. What about the gesture-based user interfaces? Any nice examples of those?", "start": 5960.318, "speaker": "SPEAKER_11"}, {"end": 6586.6114102500005, "text": "Yeah, so what about gesture-based stuff? I talked a little bit about some of those things, and I think there's a number of things to consider when we talk about gesture-based interfaces, so let me find some stuff. So one thing that I talked about... Oh, here we go. One thing that I talked about is understanding how people actually use devices and making sure our designs work with that. So just a couple of fun facts, stats, whatever that can help inform you. Number one, if you look at smartphone browsing behavior globally, you'll find that most of it happens in this vertical orientation. And this is for people browsing web pages and things like this. So what we take from this is trying to think about what is the ergonomic consideration of a gesture, right? What position would make gestures actually work and are comfortable? And I also talked a little bit about one-handed use. If you go and take a look at how people use devices, there's a couple common patterns. There's the one-handed use. There's the, I need help for my one hand with my other hand. And then there's our friend, the Crackberry Prayer, right? analysis of people doing this. This comes from 1,100 records of people out in the wild. We see half the time, it's one-handed like this. About a third of the time, it's like this. And much less so are they doing the crackberry prayer. Now, if you drill down a little deeper, what you actually find is this two-handed use comes in two flavors. One is one finger like this. The other one is a thumb like that. And this breakdown, almost three-fourths of the time, they're like that, and 28% of the time, they're like that. So cumulatively, what we see is 75% were really thinking about one hand and a thumb. And I think that forms a lot of the basis for how to think about which gestures are comfortable and easy to use. To illustrate that, I have a video of me stalking a woman on an airplane. So here I am filming her without her knowing. I'd probably get arrested for this. But you can see how she's just using the thumb here. And this clearly looks like Facebook. So she's going to flick, flick, flick. And then she tries to tap, tap. She does the two-hand thing. Tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap. Screw it. It's back to flicking. I like that video because it illustrates, A, that I'm creepy, but B, It shows the behavior I was talking about there in the abstract form very realistically. It's that one hand, one thumb. You saw her put her additional hand up there so she could get to the higher edges of the screen. And then the other thing you noticed is touch target sizes, pretty broken in that experience because they're probably just small link-based things in tiny text. and very difficult to hit. I think Craig talked about the scientific chart around gestures, and I have some stuff on that somewhere I could probably dig up. But in general, you want to make sure you have the right touch target sizes for hands and design for that accordingly. Now, what does it mean when we design for these kinds of use cases? Let me see if I've got a nice image. Probably do, but let's see here. And for reference, what I'm doing is scanning like 600 slides. So sometimes it takes me a second to find the appropriate thing. Let's open up another deck real quick and find it there. One of these will be it. Let's see. Not that one. I got a nice video of making use of those types of gestures and illustrating how they can work for you. So give me a secundo. See, I should have some jokes prepared for situations like this, right? Where I just sit there scrolling, scrolling, scrolling, awkwardly. And you say, what the hell is he doing? And I say, I don't know. All right, one more deck, and then I will just jump to this other piece. Screen, screen, screen. All right, if this isn't it, I give up, and I'll show you some other stuff. OK, I give up. I'll show you some other stuff. Actually, we can maybe talk about this. This is somewhat related. Let me show you this and then I'll switch over to the other thing. Some examples of how we can account for this kind of behavior. For example, here, this principle that I've talked about for a while, and excuse the dated example of the old style here, but there's A general principle that, and I think this got mentioned a few times yesterday, how typing is hard on mobile. I think Ani talked about this a bunch of times. There's a number of things that you can do to make typing easier and also take advantage of gestures for it. So here when you're creating this guide, there's default text up there, there's how to, and then there's these little buttony experiences where you can just tap install. So how to install, and you're off to the races. So we generally have this principle of try not to let the keyboard come up. Is there other ways that people can do things without a keyboard? And Google Trips. has some nice examples for this. So when you are searching for a flight destination, for example, we're going to default the airport closest to you using geolocation. So you don't have to enter the airport closest to you. We're going to give you suggestions of what are the most popular destinations from where you're at. So from San Jose, people usually go to Las Vegas. So again, you can just pick one of those things by default. And you don't have to type in your to destination. If you want to pick an airport here, again, rather than getting you to type, you can just hit that little location icon. It'll give you other airports near you. So if you want to change from San Jose to, say, San Francisco, you can do that without the keyboard coming up. So there you go. Now, if you want to enter a destination, you can tap that map icon. And again, you can scroll all the way around here and find a destination. These are kind of the most common places things go. And you can just tap like that. And it fills in that information, starts to tell you how much it's going to cost. And you're right back. And you're right back here. And now what we've done, you can see there's already results coming in. So if one of these suits your fancy, you're ready to go because it's sort of defaulted into a... March 8th through 12th thing, but you can tap on that date. And when you're in the date, here there's a nice big scrolling area, which again, you can kind of use with one thumb. So you can tap on 8 to 12, or however you want, and contrast that experience. So we'll show it in video form here. So I tap on date, and I can just scroll through the months really quickly and easily. I can tap the 3rd, and then I can tap the 13th or 13th to the 1st. Tells me how much it costs. I tap done and I've already picked my dates through this nice fluid one thumb kind of UI. Contrast this to something where somebody ports over like a desktop pattern and this is kind of your standard desktop widget where you flip between the months, right? And you can pick March 20th and then you tap again to get your return date and now you move through the months this way and you can go back here and tap a date there. In this case, we'll tap February 7th or February 20th. And now we're departing March 20th and returning February 21st. Anybody see a problem with that? But so this previous UI did a couple things. A, it took advantage of this one thumb kind of behavior, right? Scroll up and down, tap ones, tap the other. But it also prevented errors from happening here. And so this is, I think, thinking more for this kind of UI as opposed to porting over your desktop UI, which I feel this is an example of. And hopefully this example illustrates kind of the contrast of, let me just take my website, shrink it to a small screen versus let me think about the reality of gestures. And there's other things that happen here. If you want to change price or time duration, you get a nice slider, which also is sort of optimized for those kinds of interactions. So you can flick it over here. And when you add all this stuff up, right, that's about seven different things that are being done in this small form UI to try to keep you off the keyboard, try to prevent you from typing. Use location, use the map, use that date picker, use sliders, all these kinds of interactions that are trying to help build for the reality that I illustrated earlier, which, again, looks something like And this is one of the things that I think we should always keep in mind, which is what is the reality of use? What are people actually doing? Similar to those short bursty interactions, the number of times they interact. How are they actually holding the phone? How are thumbs sized? What can they do? What can't they do? All that can inform what we do with design and gestures. Sound good? OK, he gave me the thumbs up. All right, where else should we throw the microphone? I saw there's one behind you. Yep, two behind.", "start": 5969.232, "speaker": "SPEAKER_05"}, {"end": 6595.999728243903, "text": "Yeah, could you talk about using drop-down inputs and not using drop-down inputs?", "start": 6595.249, "speaker": "SPEAKER_09"}, {"end": 6603.339102687501, "text": "Can we talk about drop-downs and not drop-downs?", "start": 6603.095, "speaker": "SPEAKER_05"}, {"end": 6608.033720960228, "text": "Especially on stuff that's not like when you can't use location, but a set of... We have to pick one of five options or one of 50 options and alternatives to using a drop-down.", "start": 6606.518, "speaker": "SPEAKER_09"}, {"end": 7395.8368906176465, "text": "What happened? Ta-da! OK, let's talk about dropdowns. Just so happens I have 50 slides on dropdowns. I don't know how that happened. I have this, so let me back up half a second. A lot of the ways I think about doing design is under this notion of a consideration, right? As opposed to hard and fast rules, like this is the way you have to do this, or this is the answer. Instead, I like to present things as considerations, which is stuff to think about as you're going through the design process. While you're building a mobile experience, what kinds of ideas or aspirations should you have? And one of these design considerations, aka something to think about while designing, is treating dropdowns as a last resort. And why do I say this? So let's look at, in the US, picking a state. So you tap the state thing. And if you live in Alabama, this is pretty much the only good thing that happens to you. Because you get to use these drop downs of just two taps. For everybody else, they have to scroll and find their state, and then that could be multiple taps, and then they have to tap done. So it's at least four taps in most cases. Kind of painful. And remember this example I showed earlier, sometimes drop-downs get used to an extreme. So in this form, departure date is three drop-downs. Return date is three drop-downs. Number of passengers is a drop-down. Cabin preference is a drop-down. Country of residence is a drop-down. Drop-down, drop-down, drop-down, drop-down. And many people's design processes, this is the thing they reach for first. It's like their hammer, or their nail, or whatever. They always reach for the dropdown. But dropdowns, in many cases, are quite hard to navigate, as we saw earlier, especially on desktops. If you've ever gotten into a long dropdown, try precisely picking that thing out of the list. You can't really do any editing. It doesn't support hierarchy, and on and on. So I showed this pattern earlier where you type a zip code and you get back a city and state. In some cases, you may get back more than one. But instead of presenting them as a dropdown, we can present them as a little stepper. And so a stepper would allow you to move through those options quickly and easily with a nice little touch gesture. And so a stepper is one alternative to a dropdown. And what a stepper will do is move things up or down. It'll sort of change a value through a consistent set of adjustments. And you can adjust the stepper to actually have an input value inside if you want. And there's lots of opportunities where these kinds of interfaces make a ton of sense. So here, when you're picking a number of travelers in a booking form, you can just go, yeah, two. The most travelers you can have is eight. So in worst case scenario, you'll tap this thing seven times. Here's another example. Instead of using a dropdown, there's this multi-picker menu for the number of children, number of infants, which maybe it's easier just to do something like this. And if you look at Airbnb, they used to have guests and beds and bedrooms and bathrooms all done with dropdown menus. And now they've moved over to these little stepper controls here inside of their Android app. Now, there's good and bad ways to use steppers. Just there is with every single UI element. So here on American Airlines, in their redesigned form, they do use a stepper. if you look at this thing, but I think there's a couple issues with this one, which is A, these things don't really look actionable because they're so light gray and off to the sides, right? It doesn't really feel like an actionable UI. And it's so separated from this. And then if you go into this mode, if you will, let me just hit that, you enter like stepper mania. And again, if these things felt a little more like actual stepper controls, I think it would be better. But they kind of feel inactive, and it's unclear what's actually happening here. Now, another place that we saw dropdowns being used was sort of airport class of travel. So in this case, economy of restrictions. And if you tap on that, then you can see all of your options, including business class and higher. Business class and higher. First, premium infrastructure. By the way, do you think there's lawyers that work at American Airlines? I'm pretty sure a lawyer wrote that copy. Contrast that to what happens on British Airways, where they actually use this thing called English to describe There are travel classes, economy, premium economy, business. But this is a little action sheet. So this pops up from the bottom, and you can pick the type of class of travel you have. And an action sheet is another alternative here. This is kind of that bottom menu thing I showed you earlier. So you have a set of choices related to some task. It comes up from the bottom. You can have a cancel action. Works real nice. Looking at United, the way they ask class of travel is you tap on that and you come into this full page view. Which, though they're using English, this seems kind of excessive, right? But United, it turns out, is super into lists because even picking the number of passengers results in a list. So they love them some lists. Contrast the way all those folks ask for a class of travel to the way this site does it. I think this is Hipmunk, if I remember correctly. And here they've reduced a class of travel down to a series of segmented controls, or in the Android world, radio button groups. So a radio button group is a set of related but mutually exclusive choices. You generally have five or less of them. And it's also a nice contender for this kind of stuff. Now, we looked at dates earlier, so I'll just sort of jump through dates really quickly. But in general, these kinds of desktop date pickers don't work nearly as well as something that's optimized more for mobile. And you're starting to see more people adapt to these kinds of controls. Although, I don't know why Kayak makes it look like a real calendar. This looks like a printed calendar here. And I don't know why United inserts this big gap here. But increasingly, people are moving more towards single calendar things, which we'll see in a second. And once you start thinking about all the different places and ways you can replace dropdowns, there's a whole world of opportunity. You can use switches. You can use sliders. You can use segmented controls. Switches will let you do on-off type things, mutually exclusive choices. Sliders will let you move through a range of stuff. There's good and bad ways to use each of these controls. For example, I wouldn't necessarily use a slider to pick the number of people in a hotel room. And you can style sliders however you want. So there's a whole slew of things available to you, right? steppers, switches, segmented controls, sliders, date pickers, dialogues. A lot of these controls are built for touch interfaces. And they're almost, you know, they're available in some ways on desktops, but a lot of them have been built with touch target sizes in mind. And coming back to, let me just jump a couple slides to get to the point. Let me do it a faster way. Where are we? Here we are. At least I thought I was over there. So let's look at these things in action. So what's the point? Here's a bunch of different ways of doing what we did with dropdowns. We wanted to see, are these actually better than dropdowns in many cases? And so what we did is we took a bunch of apps for booking flight travel, and we specifically picked examples that use different controls to ask different questions. And we compared which controls worked best for each question. Because what it turns out is these are your list of options, but different types of questions involve different types of answers. So different controls are better fit for different types of questions. To illustrate this, it turns out choosing travel dates on a single calendar was easier and quicker than having these two calendar things. So this pick-your-travel-dates UI that I showed you worked really well. For picking number of passengers, the most efficient and easy-to-use thing turned out to be the stepper. This resonated really well with people. When it came to picking class of travel, the action sheet worked quite good. When you tap on it, up comes your options. But then also, the segmented control worked really well, too. It was almost like a tie there. And here, people kind of appreciated seeing all of the options front and center. And so, cumulatively, all these things, I think, really add up. If you remember the image I showed earlier, when you've got this drop-down written form, your first instinct is this is gonna suck. Whereas if you have a form that uses the appropriate types of inputs for the kinds of questions you're asking, then you can really make something that feels like it'll be a lot easier to do, right? A lot more approachable. This is pretty easy. All my stuff is right here. I can jump in here and interact with it. Now, the question I always get asked about drop-down apps, about drop-downs, is, well, that's OK for easy stuff, right? It's easy to find a flight that puts you in a metal tin can that'll transport you anywhere in the world in hours. But what about complicated things, like enterprise-y stuff? I think the same rules apply. In fact, this list of what makes drop-down menus hard to use I got from a company building enterprise apps. And what they illustrated is how they moved to a new series of controls to help people manage their invoicing system. And so they installed this system called the Picker. And this runs across their mobile, tablet, and desktop UIs. And if you hit the invoice due date, What you actually get is a little picker control that allows you to pick when things are due. You get a bunch of default options, so you can do this quickly and easily. Or you can go and set a specific date, which, again, is quite hard to do with a dropdown menu. You can pick a specific date, and then it'll come back and fill that in. And because they built a responsive design, this works in kind of a layered fashion on small screens as well. Looking at another one of the elements inside of this form, if you want to change the units in the invoice, you don't want to have pieces. You want to have something else. Not only can you pick between the defaults, but there is a search field at the top. So you can go in and type kilowatt. You can type or tap kilowatt per hour. There it is. And because they have these custom controls, next time you tap it, kilowatt hour sits at the top and the list gets updated. So it's an example of how alternatives to dropdowns can be applied to even things that are, quote unquote, more complex. And the reason why I harp on this stuff is, again, all these other mobile optimized controls can really help reduce input effort, make things feel more approachable, and basically get to the right question and answer pairing in terms of UI. And yes, there are situations where you might want to use a dropdown, but I prefer to treat it as the last resort. I can't do it with a stepper. I can't do it with this. Let me come back to the dropdown as my last choice. OK, I got another thumbs up. Yay. All right, where should the microphone go next? OK, it looks like in the middle here. Over there? Yeah? OK, cool. We'll make our way down gradually.", "start": 6622.851, "speaker": "SPEAKER_05"}, {"end": 7408.633085785714, "text": "Luke, question on contextual design. Google's been working very hard on answering questions on the first page of the search results. How would you design something that I haven't asked yet?", "start": 7399.144, "speaker": "SPEAKER_01"}, {"end": 7411.778540071428, "text": "How would I design something that you haven't asked yet?", "start": 7411.652, "speaker": "SPEAKER_05"}, {"end": 7420.3313333333335, "text": "I'm standing at the bus stop getting the phone out of my pocket. I want to see what time it takes until the bus arrives. Have you been working on that?", "start": 7414.314, "speaker": "SPEAKER_01"}, {"end": 7569.686200057144, "text": "Yeah, how will I know what you're going to ask? I have this mind reading device implanted in my ear. I think there's a bunch of signals that we can use. Context, though, is a very challenging thing to infer. And so the question is, can we kind of infer what people think or what they're trying to do and maybe have preemptive interfaces? It's a very nice aspirational pattern. And I do have something somewhere on it. Yeah, I do. Let me bring this up and show you. It's inside this other deck. So give me half a second. The problem with inferring things is many times we are still wrong. We just don't have enough data to know necessarily what's happening. And you can see an example of this. If you go back to early days of mobile, did anybody ever end up on a mobile site that thought it knew you were a mobile user and only showed you the things that it thought were good for mobile users? Remember this? And I was like, here's the limited dumb dumb thing for you, poor mobile user. If only you had a large screen device, I could tell you everything about myself. And so it kind of inferred that, oh, someone's on mobile. They're probably on the go. So we're only going to show them this and this info. The reality is people will use their mobile devices everywhere and anywhere. You're all using it now. Are you mobile? Or are you just farting around on Facebook? I saw a couple of guilty as charged faces. And so even though we know you're stationary and indoors and in an office, we don't necessarily know 100% your intent. So let me show you a way of dealing with that. And I think I have a nice series of images, if I can get to them. There we go. Now, this is the way I used to always illustrate what I was talking about before when this was an issue. Luckily, this isn't so much of an issue, because it used to be when we thought about taking our websites down to mobile, the question we had was, well, what do we get rid of from our website so we can make mobile websites? What sort of limited content and features will we get? And what you ended up with stuff is having Marriott, which has this big website about all of their different products and their destinations. And then you had mobile version of Marriott, which was this simplified dumbed down kind of thing. And when you tried to get here, so here I'm going to tap getting here.", "start": 7421.298, "speaker": "SPEAKER_05"}, {"end": 7579.095232142857, "text": "Mm-hmm.", "start": 7573.013, "speaker": "SPEAKER_09"}, {"end": 7842.728585941177, "text": "Or as it turns out, on the desktop version, they had this thing called Airport Shuttle Service that would actually tell you how to get to the hotel. But that wasn't on mobile. because they had cut that part out and what have you, even though it's actually technically useful information for mobile. So that's kind of an illustration of what used to happen. And hopefully, that doesn't really happen that much anymore. I haven't had to talk about that in a long time, which leads me to think maybe it's OK. But let me show you the other piece of the coin here. And I think I have a nice example. If I can find it. Almost there. Yes. Here we go. Okay. So here's a nice example of adjusting for context without denying people information. So if you look at this children's museum website, let me back it up and do it again. You can see that they have up front this big visit us, call us, and find us, and hours open information. There's still a menu in the upper right. And the menu in the upper right, if you tap that, will get you to all their information. You can go see the exhibits, education stuff, birthday parties. Now when this same website is displayed on a larger screen, you can see contact us, find us, and the hours kind of get reduced in visual presence. They're still there, they haven't cut any information away, but they change the priority of information. So when you're on this smaller screen, chances are more likely that you're potentially trying to get there, right? And so therefore they prioritize that info, but they still have the ability for you to fart around on the couch and see the specials, look at their education programs, explore the content, and you kind of go this way. So the question always comes up, well, don't people use these devices in different ways? And if you look at device usage, I've got a bunch of slides. These are sort of older ones, but I think the message is still the same. Desktop usage tends to spike during the middle of the day when people are at work. And again, they're farting around. So you'll pretty much see this pattern all the time for desktop. Tablets, in general, have kind of an evening spike. So like the morning spike and the evening spike happens on tablets, usually more of an evening spike in downtime. This is now specific to Financial Times users, and you can see smartphone behavior happens throughout the day, but here there's a big spike. And for the financial times, maybe it's people on their commute. Maybe it's people checking in on the markets as they open. Maybe there's a flurry of activity right there. Maybe people are grabbing their phone and seeing what's happening in the morning. And then in the evening, jumping back on smartphones. This is BBC on the Olympics. You see the same desktop spike. You see mobile's pretty big all the time, and then fills in the space on when desktop's gone. And you can see tablet again has this evening spike. And I think this quote kind of sums it up pretty nicely, which is PC and tablet usage, desktop and tablet usage have really big spikes, but mobile is sort of anywhere and everywhere. So it's always on in many ways. And while we try to think about context, when we think where do mobile searches happen, most of them don't happen when people are actually mobile. They happen at home and at work. So while inferring context based on what kind of device and where people are can sometimes help us, many times we can actually be wrong. And frankly, I just don't believe right now we have enough information to really dramatically change experiences based on what we think somebody is doing or isn't doing. We're getting better, for sure. We're getting more signals. But it's not at that kind of, oh, yeah, that must be what this person is doing state. Cool? OK, where should it go? I'm going to throw it to the center there. Nice. I think that was nice. I couldn't tell. I just heard the.", "start": 7584.307, "speaker": "SPEAKER_05"}, {"end": 7845.197398375, "text": "Yeah, I think he's fine.", "start": 7844.323, "speaker": "SPEAKER_12"}, {"end": 7848.9169600000005, "text": "Excellent.", "start": 7848.904, "speaker": "SPEAKER_05"}, {"end": 7873.350992367647, "text": "So you encourage us to use this more natural way to, for example, log in or register. And generally use natural gestures in our products. But what if our customers use multiple devices? So how to make iOS, Android app, and desktop experience compatible.", "start": 7850.345, "speaker": "SPEAKER_12"}, {"end": 8239.953776529412, "text": "Yeah, I think there's all kinds of really useful ways to start thinking about multiple devices. And this data also is a little older, but I think it'll... give you a sense of how much people do it. So in the US, half the people with a laptop also have a smartphone. A third of smartphone users also have a tablet. 13% have all of them. So there's a lot of cross-device kind of usage. And 90% of people that have More than one device will use devices sequentially. They'll go from the phone to the laptop, to the desktop, back to the laptop. But then also, a huge amount of them do it not sequentially, but simultaneously. So here's the amount of people that use these devices simultaneously. While the TV's on, they're on their smartphone, or they've got their laptop. on their lap while the TV's on. And regardless of whether or not these numbers are still accurate today, the general pattern is still accurate. People will move between devices, although use devices at the same time. Which leads us to think about how can we design for cross-device usage, right? How can we think about experiences that take advantage of this? And there's a couple patterns to think through here that may give you some inspiration. So one is this notion of access. And access is simply making sure that what you have is available across these different devices in the appropriate way. And the reason why this is becoming increasingly important is because people can transition between these devices much more seamlessly. So for example, if I'm on my laptop and I go to my website, I click on this link, I go click on this other link, I click on yet another link, and I just navigate it around this thing. Then when I fire up my mobile phone, you can see here with Chrome, we're actually showing you that laptop where you were just at and how long ago we synced. So you can not only tap on that page and be exactly where you left off, but the back button actually works. And you can go and follow that whole path that you just took on your desktop on your phone. And so making sure that you have an experience that's available across these different devices makes these experiences work. If you have like a separate mobile site or a separate tablet site, the URLs don't align, this kind of stuff can break. Another example is flow. So content, that's probably not the only thing we want to move seamlessly between our devices. Processes should be able to move pretty seamlessly as well. So with this, if you're in the middle of a task, you can just pick up right where you left off when switching tasks. Again, I'm on my laptop. I search for a destination so I know where I'm traveling. When I open up my Maps app, in this case Google Maps, and I tap search, there's that location I just looked for and I can take off running. So you can just pick up your flow right where you left off. Control is another interesting pattern where one device controls another, often remotely. And so here we built a, oh, actually I thought I had a video of this. Is this the video? Let's see, maybe I just skipped through it. So we built an experience with Microsoft a while ago where you can sort of talk to Xbox and you can connect your phone to a big screen and you can see I can actually be voting on things. as they're displayed up on the TV. And the TV interface has a very different UI than what I have on the desktop. And I can watch TV in the one screen. And on the other screen, I can see while I'm voting here, I can go and do a search for Futurama. And now all the Futurama questions pop up there, and I can interact. But you get the idea here, right? One device is controlling either the other device or part of the other device. And the last one is a push, and this is the one I think is actually most interesting and perhaps underutilized. So eBay has a nice example of this. If I'm on my laptop and I'm listing an item for sale, I can say I'm going to grab an image from my phone. A little notification pops up on my phone. It opens the eBay app and jumps right to the camera roll. And so I can hit the camera roll, and I can use my phone, which is always in my pocket, to take a picture of the thing I'm selling. So I just hit that, take a picture. It gets saved. I hit Done. And then automatically, from having taken that picture, it will now instantly show up on the desktop without me doing anything. So it's a really nice example of these devices working together. This has a camera that's always likely with me. Getting pictures onto the laptop is kind of hard. It's sort of a pain in the butt. So having this seamless integration through a push notification, it gives me an opportunity to use those devices together. So hopefully that gives you some ideas on how we can think cross-device. Again, the general patterns are make sure your stuff's accessible. Responsive design is a great tool for that on the web. We can talk a little bit about how to make a consistent experience across native platforms as well. Flow is this notion of processes. Control gives you a chance to have one device control the other. That's potentially more rare. But this push scenario, I think, is the most exciting one, where you can use the best part of each device together, cumulatively, to create a overall experience that works well. OK. Answer your question? OK. Three thumbs up today. I'm doing awesome. Where should we throw it? Looks like right behind you. Easy throw, yes.", "start": 7883.032, "speaker": "SPEAKER_05"}, {"end": 8253.811188323529, "text": "Hi. Hi. So you talked a lot about the checkout process, how to make it simpler and smoother. Do you have any tips for what happens before the checkout process? So for example, making it simpler to choose from products or boosting the motivation to buy and so on?", "start": 8241.404, "speaker": "SPEAKER_07"}, {"end": 8422.207880166667, "text": "Yeah, so in terms of like search patterns and the like? There's a whole series of things we can discuss around that. I'm trying to think what's the fastest and easiest way we can talk about it. Let me scan for half a second. I'll look at one thing here and see if it helps us. Oops, that's the whole talk I showed this morning. We don't want to reference that again. OK, so here we go. One more deck open recent. OK. So I think there's a lot of opportunities that we have to improve overall browsing experiences in a number of different ways. Right now, we're kind of dependent on search and browse as our primary mechanisms. But for instance, whenever you go and represent any kind of like hierarchical list for people to find things, there's an opportunity to actually get them invested in what's inside of each category. to help them make their decisions appropriately. So for example, this doesn't necessarily apply to shopping, because this is in the finance space. But contrast that browse path on the left with this one on the right. You'll have a sense of what's inside. You have a sense of some of the activity. And you know why you should access any of them. And then here, this is sort of the difference between kind of a static browse a category list and what you can do to make that a lot more dynamic and engaging. So here in the books department, not only is there search within a category, there's actually merchandising that drives you to popular stuff. There's this primary nav, this sub-nav. And so introducing these kinds of elements to any kind of guided browsing path can give people clues on where they should or shouldn't go or what they could do. So I think those are a couple things to do for sure. In terms of increasing motivation on view item pages, I feel like you guys are doing enough already. It's just me. There's like 15 million things that tell me I should buy the darn thing. So I don't necessarily have too much on that. Next. OK, you've got to really throw it down. There's like four down here. Ready? Incoming. Whoa! Oh! That was almost perfect.", "start": 8261.642, "speaker": "SPEAKER_05"}, {"end": 8460.333838460228, "text": "So actually, It is related to the last question. We are really struggling on our webshop with navigation. We have this old-fashioned hamburger menu. I thought about it when you showed the slides about the reach of the thumb. So a lot of web shops and also our web shop still have the hamburger on top, far away from the thumb actually, which is quite weird to my opinion. But I'm wondering about, I'm really curious about what opinions you have about optimal navigation on web shops that sell a lot of categories and if we should move it to down or", "start": 8425.325, "speaker": "SPEAKER_10"}, {"end": 9140.945096200001, "text": "Other ideas? OK. So let's talk about that for half a second. So the hamburger menu pattern, for those of you that don't know, it's kind of you hit a little element in the upper left, and you go to the menu on the side. So here you can go and hit the hamburger menu. You see some navigation. You hit it back. It goes away. It looks like that, and here's another example. This one over on ESPN, you hit it, it comes back, goes away. And this used to be the case on Facebook a long time ago, which is up here. In fact, the hamburger menu has become so popular, it has its own Twitter account called Mr. Liney. And he basically goes around and tweeting all the places that he gets used because it gives him a nice ego boost. Now, there's good reasons for him to be happy about this. If you go and look around, you see hamburger menus everywhere. They're all over the place. And when you talk to people about why are they using hamburger menus, you generally hear people say something like, oh, everybody's doing it, so I will too. Or you hear Facebook is using it, so I will too. And in particular, the second one, I think, is very, very dangerous because you don't know why they're doing it. I for years have gone and talked at companies where I've been really terrified going in there because I feel like they know everything. Oh my God, it's eBay. They know so much about mobile commerce. And when you come in, you realize they're figuring stuff out just like the rest of us. And in particular, the case of Facebook, though they had this hamburger menu, When they did a redesign, they found that engagement, satisfaction, revenue, perception of speed, everything they cared about, actually did a lot better when they moved to this. And the reason why they ended up sticking with that hamburger menu for so long is because their app was really damn slow and buggy and it used memory. It was essentially JavaScript creating a pseudo web browser inside of a native app inside of the OS. So they had a one or two star review for a long time because it was so crappy and slow. So they were spending all their time rewriting the app and not thinking about the nav. When they implemented an A-B testing system and they could compare the nav against the other, they realized, hey, this actually works a lot better. So be careful when you look at what other people are doing and assume reasons why it is that way. It could be God knows what reason. You know why the hamburger menu is on that person's site? Because Bob the developer and Jane the PM hate each other. And Bob refuses to implement the new now. And Jane escalated it to her boss, and Bob's being such an a-hole. You don't know. Human beings are human beings. There could be a gazillion reasons. So it's always best to go and get your own opinion on things. And I went and talked to a bunch of people around the hamburger menu. Here, some people got it, thought it was an Apple thing. Poor Danny at AT&T found people have no idea what they are. That's a shame. And then you can learn more through these qualitative things, talking to more people. You can run quantitative tests. James Foster ran this quant test where he took the hamburger menu, compared it against the hamburger of the word menu, compared it against the hamburger inside of a button type thing, and found adding the word menu helps people understand it. But making it look like something you can actually tap helps more people understand it. Shocking, I know. Then he took the winner of round one, and he tested it against the Word menu. He tested it against the Word menu with an icon, and he tested it with just the Word menu. And what he found was using words instead of cryptic symbols tends to improve things. Shocking, I know. He also found that having the icon in the menu doesn't do that much. And this one is the most funny to me, because it turns out like 22% of people need a buttony thing to understand something's tappable. Because before, remember when you put the buttony thing, 22% more used it. When you took away the buttony thing, those poor 22% of people were like, oh my god. So then he went and ran a grand finale. He took the hamburger menu in the buttony thing, the word menu in the buttony thing, and he ran it against almost 250,000 unique mobile visitors. And it turned out the word menu outperformed. which is cool. Now, of course, all of this matters your specific context. And I'm going to reference Booking.com because they brought this up. And I think somebody, maybe it was Craig, quoted from them. When they ran this test, they found no significant impact between hamburger menu and the word menu. But it turns out, what is behind the hamburger menu on Booking.com? The ability to change the currency in which your results apply. your ability to create a profile. In other words, this is not critical stuff. And so in their specific user base, it may or may not matter that people even access that menu. How often are you changing your currency settings? Hm, doesn't matter so much. In other situations, and by the way, I think if they had run this test, it'd be different. But anyway, in other situations, it matters a ton. So let me skip this slide and look at this slide. For a time, what they did when they moved to the hamburger menus, they took all the categories of their newspaper, world, politics, entertainment, sports, they hid them behind this tiny little thing. And guess what happened? To their metrics. Because people didn't know where the sections were anymore. Where's sports? Where's news? Where's entertainment? And they got so desperate, they had to add the word menu. And then they added this thing. New time, new navigation. Click the menu. OK, OK. Desperation, right? You don't need the qualitative studies and the quantitative studies that I showed you a second ago sometimes. You just need to see something like this, these Band-Aids. Look for the Band-Aids, and that's where you'll find the ouchies, the usability ouchies. And it's a nice illustration of we shouldn't just copy patterns, right? To come back to this for half a second, where did I put this? Because copying patterns, again, we don't know why things are the way they are. So if we come back to this, this was hamburger menu, post-hamburger menu for TabBar on Facebook. And you can see the before and after view. And they had better engagement, satisfaction, what have you. Another example, in this app redesign, they used to have a hamburger menu, they moved to the tab bar, session time increased 70%, 65% increase in daily active users. And I should pause because this is super dramatic, right? How is this even possible? Well, what happens is, before, all of this is out of sight, out of mind. And here, we're making it not only visible, but we're showing activity, telling you what's actually happening in this app, putting it front and center. So you know there's things that are meaningful to you. Sort of looking at it in the opposite direction, this app went from a tab design with a set of options around what's inside of this entertainment app to this navigation drawer pattern where they hid everything behind their icon and you have to slide it open to see it. And when they did that and they looked at their analytics, they said it was a disaster. Weekly and daily frequency was down. In fact, weekly and Engagement time was halved. Time spent in the app went down. So they lost half of their activity when they buried these different features and functions, which is a huge amount. So what does all of this stuff mean? I think what it means is we want to make visible to people what is inside of our apps, why they should use them, how they should use them, and those kinds of things. And last year, I won't repeat it this year because we don't have enough time, but last year at this conference, I walked through a whole redesign going from a hidden nav structure to a visible nav structure. And I outlined what the big benefits of that were. But what it basically boils down to is very, very similar to what you see inside this image here. And I actually think I've got even more examples of this now. It turns out examples of this keep popping up left and right. Let me see if I can find this last one. And so my advice around navigation is, first and foremost, understand the things that help people orient to what you do. Here's the other one. Let me play that. So here's another example where they went from this hamburger menu down to this bottom navigation menu, double-digit growth in monthly active users, double-digit growth in visitors, triple-digit growth in users, what have you. And so it's just front and center what the main pieces of the application are, what you can do there, and what you should do there, whereas before, everything is often hidden. And I guess maybe I can dig up one chart of what I showed last year to tempt you to go look at that video, if you so choose. Let me find it. Here we go. So when we put some things inside of the bottom navigation menu, here's kind of the chart we saw. This little chart is the test of it. And then that's the actual one of it. And I have similar charts for all kinds of other important-y things. Does that help answer that question? So this is, again, I think comes down to the trick of designing for mobile, finding the right stuff to put on the small screen at the right time. When people enter a service or when they're at the top level, one of the things that I think is the right thing to show them is the scope of what's possible and the means to get to that possibility. And so navigational systems, which hide all of that possibility and get rid of it, do a disservice to people. They don't know what's available. Therefore, they don't get into it. And as a result, you may end up losing them from an engagement perspective. All right, I don't know where the mic is anymore. Who's next? Let's go up. There you go. Nice.", "start": 8474.399, "speaker": "SPEAKER_05"}, {"end": 9153.577402688888, "text": "Hi. In the previous session, you gave the example of a newly designed credit card form, which on first glance looked are pretty complex to handle. So can you give a little more detail to that?", "start": 9141.709, "speaker": "SPEAKER_08"}, {"end": 9179.16452, "text": "Sure, on that credit card pattern thing? Yeah, I actually have a PDF where we tested it and show how it works. But let me see if I have a little more detail about the actual interaction design of it here. Wrong deck. Let's see. I want this one. No, I want this one. Yeah.", "start": 9156.169, "speaker": "SPEAKER_05"}, {"end": 9187.485329066665, "text": "So here we go.", "start": 9185.05, "speaker": "SPEAKER_06"}, {"end": 9329.7514, "text": "I got a video of it, so this should help you. So this is an implementation of it on the web. And so effectively what we do is we bring up, when you tap into the field, we bring up a numerical dial pad. So it's all just numbers. You can't type anything but numbers. And as you type in, you'll note if you enter an incorrect credit card number, it'll throw an arrow and not allow you to continue. If you put a appropriate credit card, it'll identify that credit card, switch it to the type you're entering, so it lets you know it's a MasterCard. You don't have to set credit card type. Then once you get to a valid credit card number, we just slide it over. And then you're on to the next field, where you're entering the expiration date. And again, it's just numbers. Now we show you where your CVV is. Again, it's just numbers. And this specific example also includes a zip code. So if you want to capture that, you can. You don't have to. But all of this information just gets collected right here on these numbers. And you get this real-time feedback, what have you. When we tested this with people, to give you some of the detail of what we found, number one, people were very happy to have this big numeric keypad. They responded really well to this because they didn't have to jump between keyboards and figure out what's going on. They wondered sometimes how to go back to editing their credit card number. Not that they needed to, but they brought it up. And so effectively, the ways to do that is to either tap on the number or to tap back. And both of those things work, so yay. And the one thing that they were actually pretty discontent about, we tested a couple implementations of this, including the one Square uses, where they put dots on there, and people hated the dots. Because it masked what they entered. Does that answer? Yeah, so maybe it feels a little complicated walking through it this way, but it's just like if you look at your credit card. I think I have a picture. If you look at a credit card, you're literally just da-da-da-da-da, da-da-da-da-da, flip it over, da-da-da-da-da. And it's all just numbers, and you stay on that input the whole time and put it in there. OK, we're to. This man's been patient.", "start": 9191.094, "speaker": "SPEAKER_05"}, {"end": 9406.93371904054, "text": "Thank you. I would like to talk about ethical way or like philosophical way how we design actually. Like I remember from the last conversion of your presentation last year that like spending time on mobile was increasing. And then today, you also mentioned that sessions in mobile also getting more often and frequently in the daytime. And to build more habit-forming products in the industry now, we are working with people from different fields, like cognitive science, psychologists, sociologists, and so on. And what we do as we design is that we capture the user, their micro-movements to get their attention. to show, to sell, and so on. I just want to ask, do you think building better product necessarily means that we are building a better life for the people, for the users? Because they could also use that micro moments time spending, I don't know, having a dinner, or having chat with friends, and so on. Do you think necessarily spending more time means a good thing for humans?", "start": 9330.673, "speaker": "SPEAKER_03"}, {"end": 9539.015700266666, "text": "I'm biased, but I personally think it's awesome that interactive devices are taking time away from passive devices. Because if you look here, this is in the US, how many hours of TV people spend? Four hours a day, on average, staring at a TV. Is this good for humanity? We're going to have an ethical conversation about that. Maybe that's good. Maybe that's bad. But my small biased opinion, at least as someone who has children, I much strongly prefer if my child is building something in Minecraft than if they are staring at a video. That's just sort of a one way conversation. And that's just my personal bias. Other people may have different opinions. So that's one kind of thing. And then the other piece is, what kinds of products are we making? I think there's a lot of products that are empowering, that give you superpowers, that allow you to do stuff you couldn't ever do before. And so I don't view those things as a detriment of attention. I view those things as an augmentation of attention. By using this, you can now navigate the entire planet. You can learn a foreign language. I mean, there's all sorts of things that help our lives. There's certainly things which are distraction for distraction's sake to sell attention. That's called advertising and that's existed for a long, long time. And it's not unique to mobile, right? People's attention is valuable and therefore you can monetize it. So I think it boils down to what are you making and what kind of perspective you take on it. My personal perspective is this increase in time spent away from these passive forms of screen sucking are a good thing. And we hear all the time, people just keep staring at screens. Yeah, they do. Sure as hell do, right? And it's not the one you're talking about. It's not the little powered computer that's in your pocket. OK, behind you.", "start": 9412.23, "speaker": "SPEAKER_05"}, {"end": 9553.97502084, "text": "Hello. Hi. Do you have any examples on? what works for mobile around the filter design? Around which kind? Filtering search results.", "start": 9542.562, "speaker": "SPEAKER_06"}, {"end": 9609.402235529411, "text": "Yeah. Oh, you know what? I'm probably not going to have it with me. But there is a book by a woman named Rachel Hinman. where she has, I think it's called the Mobile Frontier, and she has a series of patterns around filtering. I used to have a bunch of this stuff in my deck, but let me see if I do. For some reason, I feel like I don't have it here. I'll look real quick. But she has a series of patterns that you can use for filtering behaviors on mobile. Looking real fast. Yeah, I don't think I have it in here. But if you look up Rachel's book, Rachel Hinman, you'll see it. She's got a whole chapter on all of those, hopefully useful. OK, onward. OK, this guy's got a finger pointing at him and his hand up, so win.", "start": 9557.64, "speaker": "SPEAKER_05"}, {"end": 9616.69956312195, "text": "Thank you for the great stuff today. I was wondering what technology will replace smartphones, and when will it happen?", "start": 9614.038, "speaker": "SPEAKER_04"}, {"end": 9960.216728133333, "text": "Oh, awesome question. So for years, I've gotten this question around, what's next after mobile? Is it VR? Is it AR? Is it wearables? What's it going to be? And for years, I didn't have an answer to this question. And now I feel like my thoughts around this are coalescing a little bit from being around people smarter than me. And the things that I think are coalescing are our ability to take huge amounts of data and learn things from them. So if you look at some of the recent advances, and I know there's a lot of hype around this, but I've personally had several instances where I have been super impressed by what we've been able to do. And I'll try and find one of these things. But we, meaning Google, and the hype, meaning things like machine learning, So just very recently, we announced a couple of things that illustrate just how many parts of our day-to-day lives this stuff will impact. And I don't think I have a screen for this, sadly, so I'll just illustrate it. One thing that we did is, remember those terrible little CAPTCHA boxes that were all over the internet that you proved to be a human? We now, through mining huge amounts of data, can just know whether you're a human or a robot. CAPTCHA is dead. This is the single biggest contribution of machine learning to date, in my opinion. The other thing that we just launched, we launched a program or a system called Razor, which what this does is it, on your device, upscales images using an algorithm. So we send images that are 75% less bandwidth to your phone, and your phone makes them look awesome. through by looking at billions of different images. We write the way to upscale them using an algorithm to make them look awesome on your phone. So again, think about the impact of that. If we can cut down on 75% of bandwidth of images across mobile, across these 5 billion devices, look at the impact that kind of thing has. YouTube recently announced they had projections for 400 million viewing hours. per day or something like this, and they crossed a billion. And the way they did that is, again, through this deep learning of huge data sets where their recommendations just got awesome and they gave you content that you actually cared about. And so they more than doubled their goal. And so just with those three kinds of examples, you can sort of see how this will affect all kinds of things. And the stuff that I'm particularly passionate about, because you can hear me ranting about checkout and sign-in. Imagine how I feel about CAPTCHA. Hate CAPTCHA, right? Got rid of CAPTCHA. You don't have to do anything. And it's a perfect example of what I was talking about earlier, where we take all that complexity and do all this stuff just so you can have a normal, awesome life. So for me, that's a really big trend that is super exciting. And if you look at, again, the advances in translation, the advances in image recognition, this stuff is transformational and very exciting. And then the other trend that I think is also very exciting is cue the screen. We used to just make software for this. Now our canvas looks a lot more like this. And what kind of implications does that have? What sort of changes the notion of what is a personal computer? if you will. There are things in the home which can collect data, and the things in the home which can collect data can have output on a variety of screens. There's things which can be on you that collect data, and that data can have output on a variety of screens. And so when you start to combine those things And you start to combine other kinds of data input things, and you look at things that could be public. I think when we build this really quickly, I feel like the next definition of a personal computer is actually this mesh of endpoints, if you will. So there's things which out there are focused on input, collecting data. And then there's things that are out there which are focused on output. which are displaying those things. And instead of having input and output just in one device only, it's more of an integrated world. And your personal computer is a combination of your interactions in the car, things that are collecting data about you, stuff inside your house. This world is almost here yet. It's just not cohesive yet. And the other nice thing about this, again, if we project forward a little bit, is, in an ideal world, you just sort of pick up any screen, and it is kind of like a dumb terminal to whatever data source or service you want. Now you can just fluidly switch between screens. Oh, let me throw that up on the wall. Let me push this to my laptop, what have you. I think a lot of these things exist in micro situations, but cumulatively we're not yet at the state where this stuff is fluid, although it feels like we're getting there. So if you take this kind of environment. And then you take our ability to make sense of all this data that we're collecting and use it to help you with your life and the examples that I showed earlier and the many more that will come. It's going to be pretty damn exciting. Yeah. I think this might be the last one. No pressure. We've got to get you to lunch.", "start": 9622.024, "speaker": "SPEAKER_05"}, {"end": 9973.866837782178, "text": "You showed us the numbers on Cyber Monday versus the Singles Days in China. Do we have a clue what they are doing different to get these numbers on the mobile device preference?", "start": 9964.247, "speaker": "SPEAKER_02"}, {"end": 10131.749393375001, "text": "Yeah, so what's the difference between Cyber Monday in the US and Singles Day in China? In China, they're thinking mobile first. I mean, I think this is the baseline difference, right? They don't have necessarily this wide distributed PC base, nor do they have this mindset of, oh, desktop, right? People convert on desktop. They're just all in on mobile all the way. And so the deals they offer are mobile-centric deals. The experience on mobile for buying has been built from day one that mobile payments are easy and frictionless in their app. And again, there are examples of services that are At these higher bars, the Walmarts, Targets, the Amazons, they've made investments in mobile. They run campaigns on mobile, they optimize for mobile. And it's the combination of those things together that I think really boosts that number. In particular, Alibaba, I don't know, 110% if they did it this year. But in the past, they have run mobile-only promotions. So you only get the deals on mobile. You have to go to mobile to get it, which is a great way to drive their mobile usage up, get people used to buying on mobile, and the like. And while I think, oops, while I think, you know, we can look at these different countries, I'm going to just bring up one thing here. Oops, wrong deck. Let me switch over to this. If we look at what's happening in even the U.S., here's my nice chart. Can we show that slide real quick? Which screen is it? Do you see it? No, not that one? OK, I'll move it over. I can't see my mouse. Where is it? Oh, now you can see it. Sorry, third monitor over. If you look even in the United States, the number of mobile-only users is growing. So now nearly one in eight people on the internet in the US are mobile. only, whereas the percentage much higher in other countries and regions. But even in developed regions, this sort of behavior is popping up. OK, cool. Hope some of this was useful. Thanks for asking great questions. If you want to talk more, I'll be here for another couple of minutes. I believe we need to wrap up now so you know which buses to take and you can get to your lunch. But thank you very much.", "start": 9980.797, "speaker": "SPEAKER_05"}], "transcriptByParagraphs": [{"end": 0, "text": "Thank you. I am here to stimulate you, it turns out. I wish I had known that earlier. I would have brought some props. And actually, I do have one prop, which is audio. So why don't I try to stimulate you with my audio really fast? Tell me if this works or not. A little bit. Was it not loud enough? Can we make it a little louder? OK, let's do it again. Tell me how stimulated this makes you. Ready? Yeah, a little bit. All right. So welcome to the future.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "As I was preparing to talk to all of you today, I started pulling together a bunch of my thoughts around what's going on with mobile, where are we at, what kinds of things have we learned. And I realized almost halfway through the process that I was going back to a bunch of examples that I had pulled together about six or so years ago. And I was kind of revisiting these examples and saying, well, what has happened? Have we made progress here? And so that original talk was called Mobile to the Future.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And I sort of speculated, where will we get with some of these new technologies and capabilities that we have with mobile devices? So when I realized I was revisiting a lot of these same topics, I thought, well, what better title change than mobile in the future? And we'll see if it actually makes sense as we go here. But what I want to do today, we actually have Two parts together. There is part one that we are in right now. And part one is effectively me talking to you about things that I find interesting. We'll talk about the current state of mobile.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "We'll talk about optimizing for that current state of mobile in areas that matter for conversion. But then perhaps more importantly, I want to try and inspire us for what's next. I want to talk about the future part of things. Where can we go versus where are we? So hopefully that'll give you a sense for how I think about some of these topics, the kinds of things we can discuss. And most importantly, I hope it triggers a bunch of questions.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Because part two, we'll get a 30-minute break, but part two will be you can ask me any questions that you like, and hopefully I will be able to answer some of them. Full disclosure, I have a slide deck with 618 slides in it. So hopefully you ask me something in there, otherwise it's going to be a really awkward 90 minutes after the break. OK, so part one. 10 years ago. And I specifically use the 10 years ago moment because this year we hit a milestone.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "On January 9, 2007, 10 years ago, I would say we had the moment that most people associate with the onset of the modern mobile era, the thing that kicked off the smartphone age. And you're all probably pretty familiar with this event. This was the introduction of the initial version of the iPhone. And I think it makes a lot of sense. I found myself referencing this moment a lot, because it's such a logical timestamp. It's as good a moment as any to say, here's kind of where this mobile thing really kicked into high gear. And by the way, I see poor people sitting on the stairs.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "There's chairs up here. You can enjoy comfy things for your bosom, as opposed to uncomfy things, if you so choose. Kirk quoted this man a few times, I'll quote him one more time, and just kind of illustrating what was the intent behind the announcement of the iPhone. And what Steve said is he wants to make a product that leapfrogs what's out there, that is way smarter than any mobile device has ever been.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And if we kind of start from this point, if we start from 10 years ago, I think we can actually look at some really interesting and hopefully eye-opening charts that illustrate just how far we've been. And I'm going to say up front that I'm of the mindset that we underappreciate how much impact the past 10 years have had on our lives, especially in terms of distribution of technology and capabilities that technology brings. So let me illustrate what I mean. Steve says we want to make this leapfrog product that's way smarter than any device that's ever been out there.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So if we go back in time 10 years, which positions us towards the end of 2006. At the end of 2006, there were actually smartphones in the world. This wasn't a totally new invention from thin air. In fact, the yearly total sales of smartphones about 10 years ago was around 68 million globally. Now, that's only about 30% of the number of PCs, personal computers, that were sold that year. It's about 239 million personal computers globally sold in 2006. So smartphones, pretty small chunk of that, like I say, under 30%. And what were these smartphones we talk about?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Does anybody remember what these looked like? Let me remind us. It's our old friend, the BlackBerry Pearl, the Motorola Q, the Palm Trail. Anyone have one of these? There was a little while there where I carried a BlackBerry Pearl around and an iPhone, because, you know, I don't know, that's what you used to do in Silicon Valley. I don't know why. This is the real thing, but I think this thing's awesome, so I'm going to carry it anyway. We were all... Anyone remember the Crackberry Prayer? Are you familiar with that?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So the Crackberry Prayer is when you're in a meeting and you're... True story. Happened many times. So this is kind of what the smartphones of 2006 looked like. Pretty different from what we have today. Let's look at what happened since. This was the 10-year-ago situation. First things first, I want to take a look at what happened with PCs. I have to say before I show this chart, because I think you're going to be disappointed in this chart, the personal computer is the thing that helped get the internet into the world. It's the thing that built Google.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It's the thing that built eBay and Facebook and so many of the services that are now ubiquitous in our lives and give us so much information and help and assistance throughout the day, let us do so much stuff. So I really don't mean to diminish the importance of the PC when I show this little chart, but I'm going to do it anyway. There you go. Doesn't really look that impactful, does it? Effectively, PCs kind of went up for a little while. They peaked around 300-something million global shipments a year. And then since then, they have been progressively falling after this plateau peak.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And if you look between 10 years ago and where we're at today, it's only a 30 million a year difference in terms of sales. Not really the greatest growth rates out there, right? So if that hasn't been such a huge success, and yet digital and the internet and everything is such a huge success, well, let's look at the smartphones number. And here, by the way, tablets weren't so much a thing in 2006. They're not that big a piece of this anyway, but I lumped them together to give you a sense of the overall scale. We're now doing about 1.7 billion of these devices per year.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And some people, to me, have complained. They're like, oh, it's flattening out. I'm like, dude, there's only 7 billion people on the planet. There's almost 2 billion of these things sold per year. What do you want? Yeah, it's totally ending. Are you kidding? 2 billion. So clearly, we've got a lot of growth in terms of devices when it comes to mobile. And as I'm sure you recognize and understand, this creates a lot of opportunity. Again, if you consider the PC, this thing is what created the digital economy that many of our industries were built off of, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This line had so much impact on our planet and our lives. How much impact will a line like this have? And this growth of devices, of course, leads to growth of use. So let's go and rewind our clock one more time back to 2006. And this time we're going to look at usage and audience sizes, not at device shipments. Because maybe they're shipping all these devices and we just have filled the world with a lot of expensive doorstops, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Well, looking at Facebook's monthly active users all the way back in 2006 again, there was about 12 million desktop monthly active users. 10 years ago, or 10 years later, sorry, 10 years later, we're at 120 million desktop monthly active users. And again, you can see a similar sort of rise and fall on the Facebook side. And at this point, I'm sure you know what the other line is going to look like, but I'll show it to you anyway. This is monthly active mobile users. And that has reached about 1.7 billion people.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So it's a very similar pattern to what we saw with devices in general, which indicates to you these devices actually get used, unless they're desktops. Sorry. No, I'm kidding. Desktops still get used and are important in different ways. Pure quantity, right, mobile really dwarfs this experience, at least for Facebook's audience base. And some people will say, okay, well, that's cool for Facebook, but I'm not Facebook. Maybe these big established companies can really benefit from this device growth. But what about new people entering the market?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And I think the impact of all of this mobile stuff isn't just felt by established big companies. It actually creates amazing opportunity to reach a huge amount of people in a way that you previously couldn't. So let's, again, rewind our clock. And this time we're going to look at the fastest growing app ever up until 2005. Anybody have an idea what that is? What was the fastest growing app ever, roughly, 10, 12 years ago. Okay, somebody over there said... Somebody over here said... Unfortunately, it's neither of those, but I remember those from the .com era.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I think I still hold stock in... The fastest growing app in 2005 was our friend Skype. I knew that. I have a six-year-old daughter who, every time we ever say anything, she's like, I was going to say that. Really? Well, why didn't you? Always. She was going to say it, no matter what it was. So 630 days to hit 40 million users. This was huge at the time, right? Anybody remember how popular and excited people were about Skype? Skype changed hands three times between eBay, private equity, Microsoft. It was a really big deal.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And again, 630 days to reach 40 million users was a really big accomplishment. Now, more recently, I'm going to show you another app. This is a Super Mario Run on just one platform, only on iOS. They just released it on Android, I think, yesterday. I saw the headline on my way in. It's now on Android. So this release was only on one mobile operating system. And it took four days. to hit 40 million. In fact, this line is BS. I actually tried to give it a curve, but there's no curve. There's just a straight line on this chart. I'm lying to you, statistics.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But what a freaking difference, right? 630 days, and 10, 11 years later, four days. And so this shows that even something new doesn't have to be this established. Now granted, Nintendo, Mario, all these things, maybe it's an extreme example, but it just shows you how the velocity is increasing, the type of scale we now have to operate on. Mobile is at the point where we've reached the state of deployment. There's a platform, a user base, devices, capabilities, stuff that allows us to now take advantage of this amazing scale at which technology operates on the planet.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I'll talk about that a little bit more in a second. But before I do that, I thought it would just be fun to talk about how long it took things to reach 10 million users. So we talked about Super Mario Run. It's about, I think, less than a day actually before it hit 10 million. How about Pokemon Go? Any estimates? How long did it take Pokemon Go to hit 10 million users? Any guesses? Two hours? OK. More? Again? OK. That's good. It's not really a number, but I appreciate the effort. Pokemon Go took two days. OK. Angry Birds Space. Any guesses? A week?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Originally, the numbers I got were 10 days. Then the Angry Birds space guys contacted me and said, no, it was actually three days. So I got to trust them. I assume they know. OK, here's a fun one. Google+, how many days to hit 10 million users? Nobody's going to get this one. Seven? Somebody guesses? 200? 16. OK. Yay. Twitter. Any guesses on Twitter? 20. Just a bit off. 780. Facebook? 300? At this point, you should note that my chart is going down. Just saying. 852 days for Facebook. And so, again, it just sort of shows you how things are accelerating, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the capabilities that we have to reach people. A lot of times you hear people talking about gamification of apps and things like this. While there's certainly some bad things that can happen there, I think there's a lot of things that we can learn from the way Games engage people in the way they spread and the way people get into them. Cuz clearly, they're doing pretty well in a mobile world. Anyway, so the number of devices out there, pretty huge. Audience growth and the potential for audience growth, huge. Well, that's all well and good, but can we actually make any money, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Is anybody making money off of this mobile stuff? And you're going to notice a common theme here. Again, I'm going to rewind the clock back to 2006. And in 2006, PayPal, which I'm sure you're all familiar with, their annual mobile payments volume was under a million dollars. It was like $900,000, $800,000, something like that. That's how much they did in 2006 in total mobile payments. OK, so from this sub-million dollar number, and you'll notice my chart, by the way, is in billions. Let's look at what happened over the past 10 years.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "The first few years were just in the millions, so they didn't even register on my chart, because last year they did $102 billion in total mobile sales. So again, we see this huge spiking chart up and to the right. $102 billion from under $1 million in just a 10-year period. Pretty dramatic. And you may say, OK, well, good for PayPal, but my conversions on mobile are lower than desktop. Anybody thinking that? But that doesn't mean the opportunity isn't there.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It doesn't mean that all of these devices, all of these audiences doesn't create this immense opportunity for revenue that flat out didn't exist there before. So to illustrate, let's compare the biggest online shopping days of the year. And we'll look at the US first. So in the US, our biggest online shopping day of the year is this thing called Cyber Monday. And in 2006, you get the theme now. By the way, I had to just dig for all this old data. The internet doesn't work so well if you look for things that are older than a day. Have you noticed that?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So back in 2006, Cyber Monday, which is the biggest online shopping day in the United States, about $610 million. Pretty good. And if you follow the trend line upward, you can see that 10 years later, it was 3.4 billion. So very substantial growth. Clearly, a lot of people buying things online and quite a big deal. Now, when you look at the amount of sales that happened on mobile devices in the United States on the biggest shopping day of the year, so it's when most people are buying, you'll see it was about 30% of sales completed on mobile.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Not too shabby when you look at what is 30% of $3.4 billion. It's a lot of money moving through these tiny little screens in our hands. To look at the different side of the coin here, we're going to look at the biggest shopping day of the year in China, which was actually invented in 2009. It's pretty awesome that you can just invent these things. Cyber Monday was also invented. And so Singles Day was also invented as a holiday shopping day, but in China. In fact, the Alibaba company was really behind this. And so when it kicked off in 2009, it was doing pretty well.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And in 2016, Singles Day, $17.8 billion within a single shopping day online. And if you now look at how much of that happened on mobile, 84% of it happened on mobile. So not only is China dwarfing the US in terms of cumulative sales, but they clearly got their stuff together on mobile. And I think this illustrates the gap, which you can call this a problem. I call this an opportunity. Clearly. Some parts of this world are able to get to almost 90% mobile sales on the biggest shopping day of the year. Some of us have not gotten there yet.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But there is obviously a path to getting there. The issue is mobile commerce just isn't evenly distributed yet. But the potential, I think, is already evenly distributed yet, which makes it a lot easier to take advantage of this. Because if the potential wasn't there, then it would be a much harder problem. And getting from 30% to 84% mobile sales is a lot of opportunity for retailers. Increasingly, in the United States, more and more of them are actually now able to start taking advantage of this because they focused on it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So Walmart this year, which is a very big retailer, as I'm sure you know, they boasted they had 70% mobile traffic. 60% of their Black Friday is another. I don't know why we have these weird names. It's another big shopping day in the US. 60% of their Black Friday orders happened on mobile, which is double the US average. So if you focus on this and actually work on it, you can take advantage of this opportunity. Target, which is another big retailer in the US, said 60% of their Cyber Monday stuff happened on mobile. Again, double that average.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So if you can look at the quantity of money moving and you can start to have an impact here, you can actually make a big dent. And I like these examples because they illustrate work pays off. A year or two earlier, this was not the numbers that Walmart and Target had. And now, through a concentrated effort, they have them. And getting these kinds of improvements on mobile is key, because I think the last 10 years have been very, very transformative, as hopefully I'll illustrate for you in a second. But they've really just set us up for what's next.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so in order to do that, let me look at this chart real quick. So this is cumulative PCs now. This isn't yearly sales that we're looking at before. This is cumulative. How many is there in operations on planet Earth, the rock spinning around the sun today? how many cumulative smartphones are out there in operation. And the cumulative smartphone number is approaching 3 billion. It's probably right at 3 billion now, because these cumulative stats I couldn't pull for the end of last year. But it's about 3 billion. So PCs are about $1.2, $1.3 billion or something like that.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Mobile phones, these are non-smartphones. If you look at mobile phones, it's approaching around $5 billion. And this number is actually kind of flattening a touch, because what's happening is mobile phones are effectively becoming smartphones. And so you can expect this line to effectively get replaced by this line over time. So the mobile line becomes the smartphone line. Next line, I think, is the most interesting. These are the number of adults on the planet. And we've got about 5.5 billion out of 7-something billion people on the rock.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And I'm going to draw a line here at this point that I kept talking about for 2006, because this illustrates what has happened in the past 10 years. And one, the first thing to notice, we're talking about global scale here. We're literally talking about something that touches all the human beings on the planet. And as smartphones turn over mobile phones, and as that line starts to approach the number of adults on the planet, what are we going to do about that? What happens when all adults on our planet are now connected through networked supercomputers in their pockets?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Things filled with sensors, things filled with processing power, things connected to the cloud, able to access all kinds of data, collect all kinds of information. I think it really changes the dynamics. And this is why I say we're entering this world of deployment. These trends are pretty set now. It's pretty clear what's going to happen, unless, who knows, right? I can't predict the future. Hopefully, looking at a chart like this, you're convinced of the mobile opportunity. So huge growth in devices equals big audiences equals big potential for revenue.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And there's clearly people taking advantage of that revenue opportunity in huge ways. But what have we actually learned over the past 10 years about making things for mobile? How has our understanding of mobile design changed as we've gone up these lines and we've seen these curves grow in this manner? And that's what I really want to spend most of our time talking about. The first part here is just set up to convince you this stuff matters in case you're not already convinced. I hope you're already convinced.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But a lot of times, seeing it from this kind of perspective, I think, helps hammer the point home. And even though we've got three hours together, I can't possibly cover everything we've learned in 10 years. So I'm just going to talk about a couple of things that are interesting to me. Sorry. And in particular, what we'll talk about is sort of the current state of optimizing for mobile, what's possible today. But most importantly, I'm going to look at some kind of weird examples or some things on the fringes that you may say, huh, why are we looking at that?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But what I want that to do is to inspire us for what's next. I want us to look forward the next 10 years. Because frankly, what happened for a large portion of this past 10 years is we were too caught up in the past. We're still thinking of everything from the desktop perspective, and you still hear this. Even today, people are like, well, my desktop conversions are higher than my mobile conversions. We're just thinking in terms of what happened before, as opposed to looking today at the opportunity, and more importantly, looking in the future at the opportunity.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "That's how you actually take advantage of what's going to happen, by being prepared for it, by thinking ahead, by looking toward the future, hence kind of the the purpose of this talk. So 10 years ago, when the first iPhone was announced, things were pretty darn simple, right? There was a camera that allowed you to take a picture, and then you could look at that picture. That was it. Seems pretty crappy, right? Oh, right. I didn't take a picture. OK. It doesn't seem very compelling.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But over the years, each subsequent version of the iPhone and of iOS added new features and capabilities. So when we started from that first version, we could take a picture, we could view it. Then all of a sudden, we were able to do video. Then we were able to flip the camera around because there's two, so enter selfies. You could turn a flash on and off. There was now grids. You could have HDR. You could take photos with square boxes, right, kind of stemming from the Instagram explosion back when we got to iOS 7.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You can sort of note the design change that happened here when we moved to flat design. Anybody remember flat design? No? It used to be the only thing anybody ever talked about in the design space a while ago. I have to show you my favorite flat design image just because, I don't know, I want to. Where did I put my favorite flat design image? OK, well, this will take me half a second. But trust me, it's totally worth it. It's worth waiting for me to wade through these slides and find my flat design image. Oh, it is going to elude me. Oh, come on. All right, one more.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And I promise I'll stop and give up. OK, yes, yes. Window. All right, I'm losing you, aren't I? I swear, this joke is worth it. Just believe me. Oh, come on. I wish somebody could invent something that could take the picture that's in my head. and basically surface it on my screen? Is there something like that that exists? I think Google's working on it. OK, one last second. And if I don't find it, I will give up, admit defeat, and you can chastise me. Ah, yes. Beware flat design. Original Hershey's logo, flat design version, piece of poo. That was worth waiting for, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Please tell me, else I'll feel really sad later. So yeah, iOS 7 did this flat design thing that everybody freaked out about. Here we go. So we added selfies, we added flash, we added HDR, square photos. Of course, it doesn't stop here, right? Continuing down the trend, we now have slow motion video, time lapse video, panoramas, timers, live photos, portrait photos, telephotos, filtering, you name it. So we've gone from this very simple just take a picture and look at it to, wow, look at all the stuff you can do with the camera in your pocket.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And when we look at this collectively, I work all night on the animations for you people. I should actually say that. Let me just say that. These animations that you see, all this stuff, all night. That's all I do. So some of you may look at this animation, which did I mention I worked all night on? You may look at it and say, oh, this is a classic case of feature creep, right? I longed for the days that it was easy to use this stuff. Now there's so many things I can't use it. A very common reaction to seeing this, people say that, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It used to be such a simple thing, now it's gotten so complicated. And looking at this picture, it clearly feels like, holy cow, tons of features have crammed in here. Other people look at this process and they say, ah, this is the perfect example of building a minimum viable product. You basically build the first version, and people like it. OK, cool. Then you start investing more in it. They're happy with the skateboard. They get happy with the scooter. They're happy with the bike, happy with the motorcycle, and happy with the car. They're just happy people all the way around.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the purpose of this MVP sort of mindset is that you don't spend way too much time adding a pile of features onto something that nobody wants anyway. So you start with something, figure out if they like it. Then over time, you build it up. And I'm just kind of curious, having heard arguments on both sides, which is it? Classic case of feature creep? Who says this is feature creep? Wow, just three people? Okay, interesting. Or I guess the rest of you say this is MVP kind of thing at scale? Don't be afraid.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Okay, so seven of you, about 93% do not like to raise their hand in little surveys. I'm going to call that the margin of error here for everything that we do. I just think it's an interesting question. People will debate both sides of it and have very strong opinions on both sides. So of course, the camera isn't the only feature of smartphones to have gained popularity over the years. Does anybody know about this thing called messaging? Of course you do. And if we look at the past 10 years of messaging, again, 10 years ago, you were able to send a text message. I know. I know.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It just doesn't compare, does it? Looking at the past 10 years of messaging, we see a similar pattern. We go from this very simple thing to now you can, in addition to sending text, you can hit this little button. You can go take a selfie instantly. You can flip it over. You can do recording. You can send anything from your camera roll. That's pretty useful. You can do searches. You can go find animated GIFs. You can record audio. Send whatever you're saying. I mean, you can literally, this is, I had no idea this thing even existed.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "If you like fart around on here, God knows what you'll find. You can record a video and draw over it with like fireworky actions. All right, so you're recording a video and you can like light your head on fire and send that to somebody. I guess there's a time and place for that. Perhaps now, right? Oh my God, have you seen his animations? Not only that, there's a whole app store for things to add to messaging. There's a whole category of software that you can insert into messaging to do all kinds of things, to send pretty weddings or wedding doodles to each other.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I mean, there's a whole wedding category of messaging apps. Same story with how we share. Quite literally, back in 2006, you know how you share one of those photos you took of the camera? You email it. You would email the photo to another human being. That was how you took the picture from your phone and gave it to another human being. And where are we at today? Well, you have this whole share sheet thing, and inside of there, you can mail, message, tweet, save, hang out, PDF, cloud share, Slackify, LinkedInify, Flickrify, and that's not all.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Then you can go into the next layer of options, and you can print, and airplay, and assign to a contact, and put in the cloud, and God knows what else. It's almost a dizzying array of options of what you can do with that photo. I should note, Apple tries to manage some of this complexity. So at first, you get this top-level stuff, and then if you go through all this, you can get to more things, and then only if you go even deeper, they try to stagger some of this complexity. But the capabilities are huge.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Of course, this introduces discovery versus usage problems, and on and on we can talk about that. But let's move away from Apple for a minute and look at the same iterative process within mobile app design. So this is the eBay Android app roughly 10 years ago. Looks awesome, doesn't it? And here's an iteration, the next iteration of it. You can see that they added some things, daily deals, messages. And next iteration. Again, I've added a couple things. Right about here is where I think they bought Red Laser, that barcode scanning app.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Remember when barcode scanning was going to be the huge thing? Of course, they put the little barcode scanning icon up there, and nobody knew what it was, so then they put a red line through it. And of course, nobody knew what it was, so then they put the word scan next to it. And probably nobody used it. So it stayed there for a little while. And then I think after like version five or six, it went away. And then they kind of moved to search. But the way they did search here, nobody could see search. So that was probably a bit of a problem.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And in the next version, you can see that they start to realize nobody could use search. So they just hid it outright. Then they made it very prominent. And then I don't even know what they're doing here. Search is gone again. Here they added deal frenzies, which is awesome, I guess. And now search is back. And da, da, da, da, da. That release, I think, was just to tell you that PayPal separated. On and on and on it goes. And da, da, da, da, da, da. All night. So let's look at this progression all at once. Over the past 10 years, what have we been doing?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Are we just moving pixels around? Are we making progress? Are we learning things over the past 10 years? Or are we just putting search here, putting search there, putting search here, putting search there? Should we have search? I don't know. Let's try without search. OK. Put it here. Put it there. Put it there. Put it there. I ask this because this reminds me a lot of what happened with graphical user interfaces. Graphical user interfaces have been around a lot longer than mobile, right? Over 30 years.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "In fact, let's look at the progression of graphical user interfaces for a second. Here's 30 years of Adobe Illustrator designed for the desktop. 30 years! Are we making progress? Is this good? Did we nail it the first time? Nailed it. I'm just going to move icons around for the next 30 years. First time was perfect. Remember 87? That's when we nailed it. Are we doing the same thing on mobile that we did for graphical user interfaces? Is this the path that makes sense? Are we just applying the same processes and results from desktop design to mobile design?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Or are we taking all the things, like the iterations of moving the pixels around time and time again that we did on the desktop, are we just now, hey, let's do it on mobile. All right, now we've been doing it for 10 years on mobile. Is this where we should end up? Or is there really an opportunity to do more? And so this is kind of what I want to explore here. And this is why I think the inspiration piece and the looking out there piece that I'm going to get into a little bit is so important. Because me personally, I feel like we can do more.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I don't think the same 30 years of moving pixels around is the opportunity that mobile provides, especially at the scale that mobile brings. Sure, we'll have a big impact if we bring some of this stuff to 5 billion people or so. But how can we create new experiences? How can we really take advantage of this opportunity? And I think there's nothing wrong with iteration. I actually am a huge fan of iteration. It makes a ton of sense, as hopefully I'll illustrate going forward here. But it depends on where we're iterating to.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This is, to me, a lot of times the missing piece when people just get into this phase of ongoing iteration and testing. Where are you going? What's the vision? What are we trying to achieve? And so to illustrate this, let's look at an example. I want to thank Craig yesterday for bringing this one up. I didn't know he was going to do that, but it's awesome that he did. He outlined some of the things that I looked at over a couple of years around login.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so I kind of want to bring us up to speed around what's happening with sign-in and use it to illustrate this example of what I'm talking about. Now, you're all probably way too familiar with login screens, right? Everybody seen one of these? Some studies say people see 15 of these a day, which I'm sorry, not what I intended when I got into software design. But you may say, well, what's wrong with this? Everybody knows login screen, username, password, right? Everybody does it. Well, it sucks. That's what's wrong with it. So most people forget a password.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I don't know how that's possible, but they do. 5% to 10% of sessions require people going through that heinous request password loop because they forget. The number one request to internet help desks, the number one thing that internal companies deal with is password issues. So this thing actually sucks. It has tons of problems, lots of data out there that shows logging in is a pain in the butt. So one of the reasons why logging in is a pain in the butt, because you can't actually see what the heck you're putting in there. It's just dot, dot, dot, dot, dot, dot, dot.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "In the interest of security, somebody could be looking over your shoulder. Especially on mobile. If there's a guy in a trench coat and a mustache behind you, just go like this. There's your security. Not being able to see what you're typing is actually a huge issue when you consider what people make you do to passwords these days. So brief aside, call it a rant if you must, I am a registered member of this comic book subscription service, which I turn to to relax. Comic books are fun. They're supposed to be fun.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so I get this message one day, as an added security measure, we have strengthened password requirements. If you have not done so, you must create a new password that meets our updated requirements. I just want to read The Hulk, dude, right? And like, seriously, what if somebody reads The Hulk instead of me? Does it really matter? But whatever. I get locked out. I can't read anything. First, it took me a while to figure out what's going on, because the whole app broke. And then finally, I somehow made my way to try and log in, and I saw this message.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "All right, I guess I got to create a new password. And then come the password rules. Eight characters, one capital letter, one lowercase letter, one number, one special character, bang, hash, dollar, percent, what have you. How much fun is that? Not fun at all. And so we get forced to make these things that no human being could possibly ever remember. And then when we get told to log in, we can't even see the damn thing. So passwords get really complicated. And so it's really nice when apps actually let you see what you're typing in, so you don't make mistakes, so you can get in.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And as we saw earlier, those mistakes can be pretty costly to support. So Amazon, in their sign-in screen, introduced this feature that when you type in your password, you can tap that. Oops, let me come back. I guess I didn't spend all night on this one. When you tap this show password thing, you can actually see what your password looks like, and you can untap to look at it. And this isn't just a nice city on behalf of Amazon, it actually adds up to real revenue.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It turns out that inside of the shopping cart flow, if somebody tries to log in and they enter a password recovery loop, three-fourths of the time, they will not complete the purchase. So in many ways, any time you're basically sending somebody off to recover a password, you're bleeding money. And this show password pattern, Etsy launched it for a while on their e-commerce site. They showed that this actually decreases failed logins, which therefore helps you actually grow your conversion. So there's actual money behind this.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It's not just being nice to human beings to allow them to see your password. It actually reduces failed logins, which helps grow the amount of people that can actually buy stuff. And why does this work? Well, looking at some studies of how people interact with things, the pattern works like this. You start typing some dots, you hit show, then you can see the password, and then you can type the rest of it in. You know, some of the testing that's... By the way, all these things are linked in sources. I'll give you the slides later so you can take a look at them all.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But in testing, people, once they start typing the first character, they'll go and unmask the password, and they kind of understand that interaction to go and see stuff. Which is cool. So this is useful to help people log in, yay. But that method kind of requires some effort on the user's part. They actually have to notice that show button, they have to tap on the show icon to get the benefit of this feature. So in the next iteration that Amazon went through, they changed sort of the dynamics and now the password's revealed by default.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So as you start typing, you see your password and you have the option to hide it. and remove it. You can see what you're typing, yay, and then you can say, please don't show that. And again, this makes sense too, because if you simply show people's passwords by default, you muck with their heads. We have unfortunately trained so many people to think those little dots are secure, which that's all I'm going to say. But they think they are. So you muck with people's expectations when you just show them their password by default.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So if you just type your password like this, again, looking at data, most people do not expect to see their password as clear text. And many of them become suspicious. Hmm. So how do you fix this? Well, you do kind of what Amazon did, adding this very simple little control, this show password dialogue here, lets people know what's happening and helps mitigate this issue. This is a pretty small change relative to the way changes go, but oftentimes it's these small changes that really add up.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I think Craig made that point at the end of the day yesterday with lots of swear words, but hopefully it got through, which is little things can come together to make a big impact. And in this case, instead of thinking this is an error now, 100% of people notice that checkbox, and now they think, oh, this site is watching out for me. This is an improvement in user experience. It's trying to help me log in accurately. Thank you, website. I enjoy you. Taking this a step further, and note the iteration here. This is still the Amazon login screen.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This is like version four of just login. They now have a visible version of the password below this hidden version of the password. Again, playing into a lot of the stuff that we saw. So you can see that your password is still quote unquote secure with the dots, but you can actually see what you're typing that's default on and you can hide it if you so choose. So several iterations later, this is the UI that's out there right now. And I think this is an important thing. I think sign-in screens matter, right? Getting people logged in, authenticated, creates business, you name it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so iterating on it makes sense. First, you couldn't view your password. Then you could tap to reveal it. Then it was revealed by default, but you could tap to hide it. Now it's revealed below as you type and defaulted on, but you can remove it. So is this problem clearly important enough for someone like Amazon to iterate on a lot? But are they really taking this in a future thinking way? Is this more of the same stuff we saw with the 30 years of Illustrator? Are we just moving pixels around and getting maybe like a 1% boost here, 2% boost there kind of thing?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Are we optimizing for a local maximum, or are we really moving towards this future that I've been talking about that mobile enables? And what's the difference, right? What's the difference between iterating and actually taking advantage of the mobile opportunity? Well, following all these design updates, Amazon launched the ability to sign in with just a fingerprint on iOS. They use Touch ID, and you don't type in a password or a username or anything. You just kind of put your thumb on a little button And you're logged in.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And I think this is kind of a nice small example of what I'm talking about. What's the vision? Is the vision, let's gradually keep making sign-in better bit by bit by bit over a 10-year period? Or is the vision, no sign-in? Mobile devices have a ton of amazing capabilities. They've got biometric readers for authentication. They have built-in sensors, microphones, cameras, geolocation, digital compasses, accelerometers, you name it. There's a ton of stuff in here in terms of capabilities that we can start to use to really change the game. And so iteration is awesome.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I definitively believe this has made the Amazon experience better, probably in measurable ways. But we also need a vision, one that is somewhere far out, something that changes the game, a vision of what the future can be. And ideally, it's kind of rooted in the capabilities of what these things can do. For example, oh, we've got fingerprint sensors. OK, let's just off with a single tap. So as we go about optimizing for today, and there's a lot of things we can do to optimize for today, how do we think about tomorrow? What is this North Star? Where do we want to get to?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Taking this a little bit further, so Uber, which is a service you probably all know, some of you may like it, some of you may not like it. But they have an online registration form to create an account, make a profile, provide payment information. And it's pretty typical. You make an account and a password. You enter a profile, phone information, enter your payment info, submit to their terms of service, as you do. on the internet, and it's pretty well designed. They've taken a bunch of best practices into account here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So they have this typical sign-up register form, but they also have a complete registration and payment flow using just Touch ID, as we saw on Amazon a second ago. And here, what you do is just one tap on that button, and you immediately create an account and pay for your ride, and that's it. You're done. There's no sign-in form. There's no check-in form. There's no payment info. There's nothing. You just go like this. You're now a customer, and you've paid for the cab that's coming to get you. And when we look at this, what do you think the future is?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And this is why I sort of push back against questions like, our desktop conversion is better than our mobile conversion. It's because you're taking desktop forms and shoving them onto little mobile screens. Speaking of payments, which are this key part of conversion, let's take a step back and look at that, exactly what I was just talking about. This is the process of collecting payment info. 10 years later, 10 years after this onset of all these devices and all the learnings and impact we've had, our framing of payments is still rooted in desktop checkout forms, this kind of stuff.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Now granted, simply taking a desktop checkout form and making it work for a variety of different devices actually pays dividends. So when Walmart in Canada created a responsive design for their e-commerce site where it would now work on small, medium, and medium-large and medium-small type devices, they saw a big impact. They got increases in conversion, especially on mobile. And I think Ani talked about this yesterday for Alibaba. When they move to a responsive design, good things happen.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So just making something accessible on a variety of devices can help use, because you can now use it on those devices. That's a good thing. However, the entirety of what worked on desktop isn't necessarily the best fit for mobile. So when we look at Dell's checkout experience, you can see there's the shipping section, the payment section, the verify section, the order submitted section, all this stuff. Taking all of these things and putting them on mobile, all of it on mobile, and I really mean all of it, is not necessarily the best experience.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I mean, I love the fact that they're still collecting fax numbers here. There it is. Daytime fax number with an extension. Of course, that makes sense. And once I've filled in all this information, I absolutely want to not agree to your terms of service. And on and on and on, and so it goes. Oh, my conversion on mobile sucks. Duh. I'm surprised anybody ever filled it out. You should be counting your blessings that a human being made it through this flow. God bless that human. So there's a few, perhaps, obvious reasons this is suboptimal. For one, perception.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "If something seems onerous, I think the conversation yesterday around cognitive load illustrates this. So this looks freaking hard. Ugh, I don't want to do that. That's terrible. So processes that seem onerous to people they'll avoid, because these are onerous. When people see this, they're going to choose the easiest path forward, which is abandonment, going away. Whereas something like this, which actually captures a lot of the same information, feels, oh, I can do that. I can make my way through this.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So that's kind of one thing, taking the time to create this more focused, obvious form can go a long way to addressing the perception issue. But there's other issues. In addition to our instinctive feelings of complexity, you can actually measure the benefits. So those two examples that I showed earlier, I've been showing them here and there. And somebody went and tested them against each other. So they took that kind of multi-dropdown form for flight travel to the simplified version. And they found that completion times Huge improvement.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "People were able to make it through this thing much faster and easier than they were in the last one, which pays off. So in addition to perception, there's sort of these measurable benefits of improved interactions, completion times yay, and of course, the one we all care about, conversion rate impact. And for a while, I've been showing this Boingo redesign form because I like it for a couple of reasons. A, they sent me a redesign which had a real nice impact on their overall conversion. When they dropped 11 input fields, they got a 34% boost.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And again, same sort of decrease in effort that we saw before. But I like it more because I actually had a design that takes it down to four inputs that I thought would be a bit more effective. And now, what's really awesome is a couple of sites have gone and implemented things that look a lot like this. And I actually have real-world examples from very large e-commerce retailers. This is Staples in the United States. And they basically took a similar 22-input screen and got it down to just five. from 22 things straight down to five.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so before when I used to show this, I'm like, I can do it in four. And people were like, whatever. Now I can say, look, they did it in five. It's real. This is not a designer dream kind of thing. And so I think it's worth looking at this in depth. Again, as part of similar to what we did with login, what are the optimizations you can do today to work for these kinds of devices in these situations? So let's do that. Step one here is in this checkout. And by the way, this is guest checkout. If you don't have an explicit guest checkout, you're probably missing out.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "A guest checkout has been shown to boost the amount of people that can use things by somewhere 30%, 40%. Because in most cases, people don't want to get married to a service. They just want to get in and out. So step one is capturing the customer's name. And you'll notice that there is just a single input field. for the customer's name. And there's a couple reasons behind this, so let me dig into this. And we'll walk through each of these five fields and talk about what's happening. But basically, the principle here is avoiding splitting input fields.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And once people are in typing mode, just let them stay in input mode. Don't make them jump between fields in input mode too much. There's a couple reasons why we should avoid splitting input fields. And Baymert Institute has a real nice set of reasons around this. which I'll kind of walk through here. So people have a hard time moving between fields. Sometimes those divisions don't make sense. And sometimes it's unclear what is or isn't required. So we'll start with the bottom one here. When we look at what is or isn't required, of course, you know the asterisks, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Everybody knows the asterisks. We can talk about that later. But what's required in zip code? Both of these or one of these? Makes you think, doesn't it? You don't know, do you? You can guess, can't you? But you don't know. And this is an example of that issue that we talked about here. It's unclear what's required. So when you split input fields and you have this situation where certain things are required, certain things are optional, that split field doesn't really help people know what they have to do. And if it is required, it can lead to errors.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "If it's not required, you get the idea, right? So this is one of the problems with splitting input fields. The other problem with splitting input fields, people have a hard time moving between fields. So this person switches. Let's look at this from the beginning here. The person switches to the numeric keypad. OK, great. Now they type in three digits of their phone number. And now it auto tabs, so the keyboard goes down. So now I have to tap the field again, and they have to switch it over to the number field again. Now they're back in the number field.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "They can type the next three digits, and then, of course, it auto-taps, so it goes down, and they have to go back to open the keyboard, switch over to the number field, tap in the number, and then, finally, they get through that field. What a pain in the ass, right? And so taking your desktop forms that work on mobile and putting them on these devices makes no sense because it is a phone. We can just capture phone numbers like this. And if you really need secondary info, there's ways to capture it, what have you. But the same principle applies to first name, last name.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "If you've ever observed a usability test, you've probably seen somebody do this. If you've ever watched somebody fill in a form, I've seen this many, many times. People will see a name field, and they'll just enter their first name and their last name, because that's their name. You just start typing their name. You've probably done this, right? You go in there, you say, oh, crap. Oh, I've got to separate name. That's because your name to you is a single coherent entity. And this is the example of that earlier comment from Behmer.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "People find division illogical. meaning this is my name, I will enter my name, I don't think of my name in necessarily these parts until you tell me to. And so a single name is also more representative of global differences. There's people who have a single name, there's people who have eight names. True story. All right, so you can cover all these variable differences.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "The third reason why we're down to a single name is to avoid the next input field, which says name on credit card or name on payment method, which then you have to ask yet again, because now you split the name, so you need the name that shows up on the credit. Here, you don't have to ask it again. You just ask for their name. So we got their name. Yay. What are we going to do about shipping address? We were able to collect a name. That's good. For here, what we're actually going to use is a principle called inline validation, which is helping you as you type.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It's real-time feedback that helps you avoid errors, helps you get to correct answers. And so what Staples has done in this case is made use of the Google Maps autocomplete address API. And the really neat thing about this API is you can restrict it down to particular countries or not. Basically, you start typing, a dropdown with a bunch of autocomplete suggestion comes up, and in pretty much all cases I've ever seen, the address is there before you finish typing it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So you save a bunch of time, you improve accuracy, you minimize errors, and just by typing 712 second, you hit that button, gone. Uses geocation, helps you manage all this good stuff. So here, through inline validation, we've now collected a shipping address, and we've done our best to make sure it is a valid shipping address. And we haven't had people jump between 20 different fields to do so. Saving effort like this pays off. We did some tests not that long ago, comparing collecting city, state, and zip.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Now, granted, this is in US addresses, so different regions have different variants of what you can look up and how you can look it up in addresses. and different structures. Again, we can talk about that more deeply in Q&A if you want. But here we were collecting city, and then collecting state, and then having people enter their zip code. Whereas in this version, you entered your zip code, and we would automatically go and fetch the city and state based off of the zip code that you put in. which was cool, so we infer that automatically.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And when we compared the two experiences, we found doing those three fields, city, state, and zip, took about 21 seconds, and this took about eight seconds. And looking at personal preference, the majority of people strongly preferred doing it this way as opposed to this way, because it saves them a bunch of time and effort. So taking this sort of principle and applying it to the whole address field helps a lot as well. Now, we can use a similar technique for capturing email or contact information. And so again, we'll use inline validation here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And as you start typing your email address, we'll do an autocomplete based off of common email addresses. And you can, again, tap on the thing that is right for you. And you're off to the races. Why do we have this? Has anybody seen this repeat email address thing out there? You've probably encountered this. Why the hell do people do this to us? It's cruel. It's unusual. It's punishment. The reason why they do that is to get rid of this thing called hard bounces.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So on average, 15% of times you collect an email address, it will hard bounce, meaning it's not a valid email address, which is not what you want when you're setting up a new customer relationship. So you ask twice to make sure that errors get cut down, what have you. Our friend inline validation can get rid of this really erroneous stuff. So when you type something that looks kind of like a Gmail account, but they mistyped it, you say, did you mean that? You add it here. This sort of interaction will cut down on hard bounces by a huge amount.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You can never get rid of the people who type joe at fart.com because they hate you. Actually, what they hate you for is making them register again. I had a real funny story for half a second. I did this two days ago. I had to register to download a white paper. And I put in bob at fu.com. And they're like, that email address has already been taken. True story. So yeah, verifying email address with a real-time message will get rid of some of those pesky balances. Cool. So now we've got name, we've got address, we've got email. Last but not least, we want to collect payment info.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And there's a couple things happening here. First, I'll show you the input mask pattern. And then I'll just sort of reference, they also enabled the ability to scan a credit card. So this is kind of available now, mobile, where you can just take a picture of your credit card. I have no stats on how much people actually do or don't use this. But it's an example of an attempt to try and use the sensors on the device to make this process easier. You just point your phone at a credit card, it collects all the info, and you're done. Pretty neat.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But let's look at the input mask implementation. And the way this works is, note you're up with a big dial pad, so you're just collecting a bunch of numbers, you start typing your credit card number in, it highlights the type of credit card you have, because there's a pattern behind these, you don't have to select the type of credit card you have, so we can get rid of that input field.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "We highlight that, and then what we're going to do is scoot this thing over, and now since you still have the numeric key, you just type in your expiration date, and then based on the credit card type you have, we change the icon to show you where your CVV security code is. You type that, and without ever leaving the dial pad, you've entered all the information you need to process a credit card. We've done a bunch of testing on this, and this format is actually very successful. We tried it out on Android and iOS, and it works.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So last but not least, we've got name, address, contact info, credit card. And then we have a smart default set here, which is use this address as a billing address. You can undo that and change it if you want, but in many cases, you don't. Here's how much it's going to cost, shipping, tax, everything you need. Hit the button. You're done. All right, five input fields, done. And again, we see a lot of iteration on what is a very important issue for a retailer. Check out, huge important issue for a retailer.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "We saw them reducing inputs, doing address lookup, trying inline validation, input masks, scanning credit cards, using smart defaults, all sorts of techniques. And the impact of this was actually pretty dramatic. They had a 50% reduction in checkout time, 1,000 basis point improvement in people who started checkout, engagement went up 40%, 70% reduction in clicks, all sorts of good stuff. And there's a lot of big impact from this. But again, I ask the same question. Are we still just sort of moving pixels around? Are we optimizing for a local maximum?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Or are we really moving towards the future that mobile enables? And again, I'll say the same thing. I think there is a lot of value to doing this. If you go do this for your checkout flows, probably good things will happen to you. A lot of improvements to be made. And hopefully just showing you these kinds of improvements gets ideas going in your head. Oh, we could do this. We could do that. Why don't we try this and try it? Great. All sorts of optimizations to do and clearly examples out there of those kinds of optimizations moving the needle in the right way.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Going from 22 input field form down to a five field input form that reduces errors dramatically, that seems like a win to me. But let's go back to Amazon for a second and check out how their payment process works on mobile. So here I see a lovely little live pet that I would like. And so I will hit buy now. And then the only thing I do is I just swipe. Done. It's going to be at my house tomorrow. That's pretty awesome, right? I just hit buy now, and then I just go swipe, and the little lovely pet is at my house tomorrow. Super fast process.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Again, to echo some of the talk around psychology that I think kicked off the conference yesterday, reducing effort, reducing cognitive load, making it frictionless, fast, and easy for people to buy things. There's a reason Amazon is doing so well, because when you see something, you just got to go, and you paid for it. Now, granted, they had to collect this upfront information first, but maybe you could use that pattern I just showed you a second ago from Staples to do that one time. So that's pretty painless to get at one time. And then after that, you start thinking more like this.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And not only is this process super fast, but it actually aligns with how people actually use mobile devices, which is specifically one-handed. There's a lot of one-handed use of devices. And as devices become bigger, as our phones continue to become more fablity, things that are up high become more ouchity to use. So this is sort of the comfortable, nice-to-hit area on your phone. And so these interactions that are down in the bottom, like that swipe and tap, really works a long way. Our fingers are only so long.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And when we start to build things that are designed to be used in our hands, we have to think about how far can we, especially one-handed, reach our fingers. They only stretch so far. And looking at this in terms of market trends, you can see that these big phones, these phablet-sized things, five to seven inches, are really increasing in share at the expense of these smaller-sized things.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So when we work on designs, we really want to make sure that we're thinking about these market trends, hoping that there's some longevity behind what we build, so we don't just, again, build for today, but we look at where the market is going. Ani showed this payment request API that Google has been working on with web browsers. And I really love the philosophy of this, because it illustrates kind of what I was talking about a second ago, which is you come over to, say, a merchant on Shopify.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You hit the Checkout button, upslides this little thing that has all of this information, because it was stored in the browser. And you just hit Pay, and you're done. All right, super frictionless, super fast, super easy. Granted, browsers have to implement this. And granted, people have to go and put their money credentials and off inside the browser. But if it's there, wow, it's almost as fast and easy as that Amazon system by taking your pre-populated info and giving you this single action pop-up.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So once again, we see Amazon not sitting still in their implementation of this and continuing to iterate. We saw a lot of iterations on the payment experience. But once more, I want to ask, what are we going toward? What is the bar here? What is the North Star? Where do we want to go with all this iteration? One tap and a swipe, that's pretty damn good. Can we go further? Or can we take this? And certainly, Amazon thinks we can go a lot further. Let's look at some of the stuff that they have done recently. This is the Amazon Dash button.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This is a little thing that has Wi-Fi, it's connected to your account, and it's effectively a single-use device. It only does one thing. You know what this thing does? Orders Tide for you. So you tap it one time, laundry detergent shows up. That's it. And in context, it's actually pretty darn useful. You put it on the washing machine. Where do you run out of washing machine detergent? In the freaking laundry room, right? You don't think, oh yeah, I need more laundry. Maybe it pops into your brain somewhere, but right here and there at that moment is when you need the stuff.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So you tap it once and you're done. There's no browsing, no opening an app, no finding the app, no clicking over to search. I mean, think about all the steps this removes, right? You just go like this. OK, it's coming. Versus, oh, let me unlock my phone. Let me hit the app. Let me do a search for Tide. Oh, is that the Tide I like? I don't know. Is this the Tide I like? Oh, OK. Now let me do it. And then I get my buy and swipe. But this eliminates all of that. Press it once, and detergent is on its way. In one tap, in context.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And this is what I promised to be one of those weird examples, because you're like, I can't do that for my insurance thing. Maybe not. I know the insurance question is going to come up later. How can I do this with insurance? Thank you. You have too much money already in insurance. You don't need to do anything. Just keep doing what you're doing. You have all the money you could ever want. Don't be greedy. I'm serious. Insurance is a lot of money. But I don't mean necessarily go and implement an Amazon Dash button for your thing. But have this kind of vision in your head.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "How far can we take it? What is the North Star we should go for? Let's look at that in another example. Amazon has this new line of physical stores called Amazon Go. They boast no lines, no checkout. To me, this is an example of an awesome North Star. Instead of let's optimize checkout, how about no checkout? I love that. Our goal is to have no checkout. And if you have that kind of vision in your head, no checkout, then you can see how you might start behaving differently.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You could still go through the iteration process, but as you're iterating, you're constantly trying to get to this idealized end state, something that goes beyond what you have today, something that feels like the future or is the future. So let's look at this for half a second. Here's how Amazon works from a user perspective. And I apologize for the marketing-y music thing. I'll come back and talk after it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Four years ago, we started to wonder, what would shopping look like if you could walk into a store, grab what you want, and just go? Thief! What if we could weave the most advanced machine learning, computer vision, and AI into the very fabric of a store, so you never have to wait in line? No lines, no checkouts, no registers. Welcome to Amazon Go.", "start": 0, "speaker": "SPEAKER_00"}, {"end": 0, "text": "OK, so how does it work? You just walk in, you tap your phone on the thing, you take what you want, and you leave. They call it walk-out shopping, right? And it's an attempt to create a frictionless shopping experience. Now what's really going on behind the scenes, because that's a whole other story, machine learning, facial recognition, microphones. Let's look at this for half a second. So how does Amazon Go actually work? So when you come in, you have a little app on your phone that you scan with the sensor. It identifies that user. There's a camera.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the camera will actually use facial recognition to uniquely identify each shopper. And then throughout the store, this is the creepy part, right? Cameras see what you do, and whether that item stays with you or not. There's images captured in possession of the item, we just talked about that. There's microphones that know where you are and where people are, so they know where to kind of point the cameras. There's infrared pressure and load sensors on all the different shelves. Note when an item is picked up or picked back.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And then there's a whole bunch of kind of deep learning and or machine learning kind of systems that make sense of all this data behind the scenes to tell where people are, because literally what they're doing is they're tracking everything in the store in real time, including human beings. So they have a map of, okay, this person's over here, these items are in flight, these items are with this person, this person left with these items, what have you. A lot of complex stuff, right? Facial recognition, microphones, cameras, machine learning, all this stuff behind the scenes.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the interesting thing about this, in my opinion, is all this complicated stuff This technology is very complicated. Think about all the things that they're doing in their track, and all of that stuff is going on from a hardware and from a software perspective to make a very simple and frictionless shopping experience. And this is the trick in a lot of what we do in the technology industry, is take all this technical complexity, bring it on yourself. Take it on as a burden, and you take it. Don't put it on the human being. You deal with it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "All the human being have to do is walk, take thing, and leave. That's how you make a happy human. You take all the benefits that they get from all that technology, and you throw it behind the scenes. They don't even have to think about it. They just go and take things. They feel like they're stealing. They're elated. They're kind of nervous. Oh, my god. Whereas the machines do all the heavy lifting. Again, why I like that example of Amazon Go is because I think it's such an illustrative example of this.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Look at all the stuff Amazon is doing to allow you just to walk in and walk out of a store. No lines, no checkout, no anything. Coming back to the Dash button, this is another great example of that philosophy. This thing has Wi-Fi, it connects to your account. That account is built onto a complicated billing system. That billing system has returns, that billing system has processings associated with it. It's even connected to a much deeper distribution and delivery network where there's warehouses around the entire world that do the distribution and they work with carriers.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I mean, there's a gazillion things happening behind this tiny little button when you push it. What do you need to do? Push the little button. Everything else behind there is then working for you. And it ultimately delivers this very frictionless and natural experience. And again, when we want to be inspired by things, to me, this is kind of a nice North Star-y thing to dwell on. To stick with this theme for a second, let's come back to Apple for a minute. I think the new AirPods are a great example of this kind of moment as well.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So in order to connect your AirPods to any device, you just open the lid. That's it. You open the lid, a little thing pops up. It says connecting, and they're connected. You're done. I didn't touch any button. I didn't open any app. I'm not hitting any UI, nothing. I just flip open the lid on the case, and I connect it. Now, you want to start playing music, what do you do? You just put it in your ear. It starts playing music. If you want the music to stop, you just take it out of the ear. I know, you're all stunned. Like, no, it can't be like that.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I got to hit a button, right? It's got to be an app. I got to sign into an app. Nothing. It's this frictionless, graphical user interface-less kind of natural interaction. And this term natural user interaction, NUI, it's been batted around for a while, but the general concept is we create interfaces that are based on a lifetime of learning. Amazon Go is a great example of a natural user interface. You walk in, you pick something up, you walk out. You know how to do that. You've been doing that your whole life. Right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So here, you want music to start playing, you just put in, done. Now, if you want to charge these things, you just put them in the case. Case charges them. Pretty darn cool, right? Another example of a natural user interface we saw at this conference. Google Home, just talk to it. It'll give you answers to things, help you manage your text. You know how to talk. Most of us know how to talk. Some of us have some problems talking. But we know how to talk. So see a pattern here? between the Dash and the AirPods and the Google Home.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I think the pattern here is this differentiation between these natural user interface type experiences that we know how to use based on a lifetime of living. They're frictionless, and I think these things serve as a really great north star for where we can go. Can we design checkout with the goal of no checkout? Can we design sign-in with the goal of no sign-in? It's these kind of aspirations where I think we can do more than just incremental improvements. Because while, again, I'm going to jump back to this picture that I showed earlier around sign-in.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This is what I'm really interested in. What's that moment where we just change the situation? Where do you go from this ongoing iteration, which I'll say again, has value. I'm not poo-pooing this, right? There's a lot of value in this. But what's the moment where the bit flips and the dynamics change? And all of a sudden, registration is just putting your finger on something. Or all of a sudden, buying something is just walking out of a store. I think those moments are really, really interesting.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And just being inspired by those goes a really long way, hence why And I keep showing these natural user interfaces because I think they're a great North Star from an inspiration perspective. And coming back to these for half a second, what better example of a natural user interface than using your eyes to see? And so this new thing from Snapchat, Spectacles, make use of a camera on these lenses to give you 180 degree of what you're seeing. It's actually pretty cool. You can just turn on recording by just tapping the glasses. That's it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But then, This hardware experience is just so simple. You wear glasses, you tap them, it records what you see. Wow, super easy. The software experience is almost the exact opposite. And I had this moment hit me when I was using these things and I tried to get the video off of them and I felt like I went back. in time, 10 years. Because to use these things, to get the video from them and send them to someone, I have to take out my phone. I have to tap on an app. I have to open the app. I have to go and tap on my memories. And in my memories, I have to find the thing I recorded.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And then I have to go and flip through these things and tap multiple times to find the video. And then when I find the video, I have to tap on Send. And then after I tap on Send, I have to pick who I'm going to send it to. And then I can actually send it. Remember how I said we want to hide complexity from the user experience? This app open and UI dance is not a great example of that. And many times what we do with our software is we actually show that complexity to people. In the Amazon Go example, we're trying to bury all that complexity, get it out of there.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Don't make people worry about it. But what we do with software a lot of times is we shove it in people's faces. And why? Because we can't make a decision. We can't decide what matters most right now. And so what we do is we just throw every option at people. Oh, I can't decide what matters at this point in time. So here you go. Just take it all. This was very easy to do when we were designing desktop screens, because you just throw every big screen, throw everything you want in there. I don't know. What the hell do they want? Horoscope? No clue.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "They probably want to see this woman dying. I don't know. Maybe how about the Netflix rating system? Sure. Throw that in there. Let's throw it all in there and show everybody everything. You can't do that on mobile because the screen is tiny. And so you have to make these important things called decisions. And mobile, in my mind, is all about getting people the right thing at the right time. And so I'll just show you two simple examples of this. Let's come back to messaging for a second. So I get a note that says, what's your address?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And without me doing anything, there's two little buttons that show up right underneath the reply that say, work, home. Please don't visit me. And I just tap home. And I'm done. That's it. This is a great example of the right thing at the right time. And it's just like one tap, just like the Amazon Dash button, the Snapchat recording example. This is why I think those natural user interfaces are good inspiration points. They get us thinking like this.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "They get us to think, well, what is the thing that right at that moment, at that right time, I think somebody echoed Jeff Bezos earlier, where he said, we start from the customer and iterate outward, right? What is that customer perspective? What is the need that people have, and how can we fulfill it? Sticking with messaging, shifting over to Android, you get a message in a language you can't read? No problem. Just tap it. You get tap to translate right there. And without ever leaving the app, you can translate that text. You can even then reply in that language.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It's another example of just a very frictionless, in the moment, right time, right action kind of UI. And why do we care so much about these fast frictionless experiences? Am I setting the bar too high? Will this ever work for insurance? Why do we care about such immediacy? Many of you are probably thinking, well, yeah, OK, that's cool. But things I do can't be like that. My stuff is complicated. It can't ever get simple. And sure, maybe there are some things that can't or shouldn't necessarily get that simple. But it sure can't hurt to try and go there, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Why do I raise the bar so high for mobile design? Why do I care so much about these fast, frictionless, right time, right action interfaces? Well, let's look at how people actually use their mobile devices. Here we see mobile 3G traffic patterns. So these are different kinds of devices connected to a cellular network. And they're all on the same kind of network. The only difference is there's different kinds of devices. And so you can see laptops, tablets.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And what you notice about smartphones is there's a lot of these short, bursty sessions, a lot of in and out on the network. contrast that to these bigger, longer, more sustained sessions on laptops. And if you dig in a little bit further, Apple recently shared how often people log into their iPhone. They said it's 80 times per day on average. An on-device research study of Android found similar numbers, 76 times, and extreme users have higher numbers or so, but these are sort of averages.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So there's lots of sessions. roughly like 80 something a day, which matches kind of what we see here, right? Supports this data, lots of small singular sessions. So lots of sessions happening on these devices. And if you look at what these sessions look like, there are certainly some long ones. That is very true. Occasionally you watch Netflix, play Angry Birds or whatever. But most fall, around 55% fall under 30 seconds or less. And then the average is somewhere between like 80 to 120 seconds. So it's not a lot of time per session. Each session is really short.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "There's a lot of them, but in general, they tend to be short. And so understanding this context helps explain why I care so much about these fast, frictionless experiences. I'm trying to design for the reality of how people use these devices. They use them a lot, but they use them in small doses, quick chunks. And so having this forced concentration on building for that reality helps a ton. I think these numbers also illustrate, remember that talk about mobile performance, In this chart, what do people hate the most when browsing the web on mobile? They hate slow things.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Why do they hate slow things? Because they're only on the damn thing for 30 seconds. If it takes 10 seconds to load, no wonder people dislike slow experiences. And this is why solutions like things like skeleton screens, I figured I'd just throw this in here since we talked about performance earlier. So skeleton screens are effectively these little baseline grids that make it look like content is coming in so you get a sense that things are actually happening, but they load very, very quickly because they don't grab images. and actual content yet.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Actually, I'll show you how this works. My favorite example of skeleton screens is from Google Photos. So from Google Photos, when you load up a page, you get this grid. It sort of shows you the relative proportion of images. For example, you can see this is a panorama. A little hard to see on the screen here, but it'll identify what's a video and how long it is. So you get the structure of your content coming in instantly. It's already there. And then what we do is we gradually load in the content dynamically. Well, we try to be smart, so we'll fade in things as they come in.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And then the other thing that happens is the content actually loads where you're scrolling to, not where you've been, so that it anticipates your movements and is there and ready for you when you show up. So lots of things you can do. And all of this stuff is around perception of speed. There's a certain amount of speed that we can get from loading images and optimizing them and doing all that. But then there's a lot of things that we can do in the user interface to make it feel like something's happening to create the perception of speed.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And again, another topic we can discuss in QA if you like. OK. Coming back to why these solutions exist, mobile interactions are frequent but short. Another study that put cameras on people for a couple days and followed how long they looked at screens in their lives found it was about 38 seconds on average. So similar data again, but the reason why I'm referencing the study is because what they were primarily looking at is smartwatches.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And they put cameras on people, they watched them throughout the day, and they saw how often they looked at their smartwatches and what they did on them. And you'll find that the bar on these kinds of devices is even more extreme. The average interaction time is about seven seconds. Way more hardcore than mobile phones, right? And I like this because it offers another opportunity to learn. This is my continuing thing here. Let's look at parallel stuff to see what can inspire us and can help us go even deeper on mobile.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Because if you are designing things for a 7-second world, you're going to kick some serious butt in the 30-second world. So let's just look at a couple of these smartwatch interactions. In this workout app, they're used to you run a workout, and then it tells you you can save or discard it, and then you have to either hit one of those buttons. Sometimes you hit it on accident. What happens in the Second version is it just automatically saves for you. So a nice example of taking the burden on ourselves instead of putting it on customers. And this is another way you can save a step.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You can save some effort. You can save some thinking. Just do things for people when they make sense. You've been working out for 30 minutes or an hour? I'm going to save this so you don't freak out when you lose it later. You can discard it after the fact. Sticking with the same workout app for half a second, The first time you use it, you get this series of choices. You can either do a walk, run, cycle. Second time you use it, what it'll do is learn what you did the first time, and it'll put the last type of activity that you did at the front and make it nice and big.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So this is an example of learning from past behavior. You've done something once. We don't need you to repeat it yet again. Here you go. Shifting over to another app, this is a little timer app. And in the previous design, you had to meticulously pick how long you want the timer to run, and then you hit start, and then it finally starts. In the next redesign, what they did is they just made these super fast and simple defaults. Customization is down lower if you need it, but you just tap it and you're up and running.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So this is a nice example of setting some custom frictionless defaults. And this last one I think is, or is this the last one? Maybe the second one. This one's my favorite because this actually changed how I use this device. So if you looked at the way they did messaging before, you had to tap to get the option to reply to someone's message. And then you hit reply. And then you get a choice of how to reply. In the new version, right there below the message, you get the little microphone. Super fast and easy to actually respond.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Before, when you had to tap multiple times, it was like, oh, I might as well just get my phone. Now, you just go, boom. And instantly, it's faster and easier than actually using your phone. Again, small stuff, small changes, but it's shifted kind of the behavior. And I lied. One more smartwatch example for a second. Again, in this case for inspiration, this is one of the things that I think illustrate to me how natural user interfaces and software and all the capabilities of these devices can come together. Because right now, I can get a light little tap on my wrist.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "There's a haptic engine that sort of taps me. It goes, bup. tells me something's new. I twist my wrist up, it uses the accelerometer on the device to know it's changed position and tell me, here's what I blipped you about. This satellite is about to pass overhead and you can see it. Since you're an astronomy geek, cool. Right? So I just tap on it once, and it uses the magnimeter, the digital compass, to show me where that satellite is relative to me. And if I want to, I can tap in again, and now it uses GPS to say, here's where you are. There's that street.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You should look in that direction. That's pretty freaking cool. It's using all this stuff, the GPS, the magnimeter, the accelerometer, the haptic engine, and all this data and technology behind it. But to me, it just feels like, hey, satellite, where? Oh, right there. If you like to look at satellites. Some of you might not. My kids get a thrill of seeing the ISS fly overhead all the time, so it's pretty neat. Anyway, what can we learn from smartwatch app design, where this time to interaction is super extreme?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "We looked at ways to minimize effort, learn from past behavior, and use these sensors, all that good stuff. And all of these are really good insights that we can then bring to mobile. And so I'm just going to end with two examples of doing that, and then leave you to break, and we can talk about all this good stuff going forward. So things like pressure-sensitive inputs. If I'm looking at a list of music, and I do this thing where I just push a little harder, Right then and there, I can go add it to a playlist, download it, delete it, remove it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It's this really fast and easy shortcut inspired by a quick little gesture. Of course, I can still tap regularly, go to the page about the song, tap the little meatball menu, and then get those choices. It's certainly there because the other gesture is a little hidden, but this nice, fast, and fluid shortcut helps a lot. Here's another example of these sort of fast and fluid shortcuts. And this is a technique called fluid gestures, fluid touch gestures. So here what I do is I just press and hold on the microphone. It starts recording. I just move up and let go. Audio gets sent.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So again, I just press and hold. Starts recording as I talk. I move my finger up. gets sent. One fast, fluid touch gesture that does multiple things at once. And contrast this type of fluid gesture to something like the current UI in Maps. You have to touch Search. You go and say, I want to go home. And then it comes back. And you can say Start. It's this multi-tap, very GUI-like experience for getting things done. And if you contrast this to a fluid touch experience, you just hold down Search. Up pops a list of options. Drag up. Let go. and you're navigating away.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Again, more inspiration than anything else, but when you start thinking about these fast, fluid, frictionless interactions, you can see how you can start thinking about making things that work like that. All that aside, hopefully some of these examples from hardware, from smart watches, from natural user interfaces, from fluid touch gestures, one of those things I hope has inspired some of you, because that's the point here. And the point is we need to know the experience we want in the future so that we can work toward it today.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Mobile has created this huge opportunity for us, for audience growth, for revenue. All these devices out there are at global scale. Huge area open to us for deployment. And we can, using a bunch of the techniques I showed you today, optimize for today. There's ways to take critical interactions like checkout, critical interactions like sign-in, and make them a lot better. You can go and make these improvements right now.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And you can probably end up in a state like Amazon or Target that I showed, Walmart and Target that I showed earlier, where they're double the industry average, right? Instead of 30%, they're at 60% sales on mobile. All this stuff is possible today, provided you put in the effort. But we also need to look past today, toward the future. And if we look to the future, I think there's an opportunity to remove this friction even further, to take all this technical stuff and put it in the service of human beings, so we can just go live our lives. So we can just do what comes naturally to us.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And we use our understanding of technology to enable that. And if you have a vision like that, I think it helps with the optimization and iteration process, because you kind of know where you're trying to get, and you can evaluate your steps. The future, I think, is going to be even more seamless if we do our jobs right. The other point I'll make is the next 10 years, high degree of likelihood are going to move even faster than the previous 10.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So the sooner we get ready for them, the sooner we start thinking about them, the sooner we start aiming for them, the better off we will be when they arrive. And those are kind of the three big things I want to leave you with. So thanks. So as promised, first part was me blabbing on for a while. Second part is us talking. So we have this microphone, which I will do my best to throw properly. And I am here to answer your questions. Hopefully, there's things that are on your mind that we can talk through that will help you. And who's up first?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Either I can't see, or you're all way too shy. OK. All right, here we go. Ready for it? Oh, that's not even close. You have to ask a question. Double-handed. Didn't even work.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "All right.", "start": 0, "speaker": "SPEAKER_11"}, {"end": 0, "text": "OK.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Well, you were talking a lot about the user interfaces. What about the gesture-based user interfaces? Any nice examples of those?", "start": 0, "speaker": "SPEAKER_11"}, {"end": 0, "text": "Yeah, so what about gesture-based stuff? I talked a little bit about some of those things, and I think there's a number of things to consider when we talk about gesture-based interfaces, so let me find some stuff. So one thing that I talked about... Oh, here we go. One thing that I talked about is understanding how people actually use devices and making sure our designs work with that. So just a couple of fun facts, stats, whatever that can help inform you.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Number one, if you look at smartphone browsing behavior globally, you'll find that most of it happens in this vertical orientation. And this is for people browsing web pages and things like this. So what we take from this is trying to think about what is the ergonomic consideration of a gesture, right? What position would make gestures actually work and are comfortable? And I also talked a little bit about one-handed use. If you go and take a look at how people use devices, there's a couple common patterns. There's the one-handed use.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "There's the, I need help for my one hand with my other hand. And then there's our friend, the Crackberry Prayer, right? analysis of people doing this. This comes from 1,100 records of people out in the wild. We see half the time, it's one-handed like this. About a third of the time, it's like this. And much less so are they doing the crackberry prayer. Now, if you drill down a little deeper, what you actually find is this two-handed use comes in two flavors. One is one finger like this. The other one is a thumb like that.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And this breakdown, almost three-fourths of the time, they're like that, and 28% of the time, they're like that. So cumulatively, what we see is 75% were really thinking about one hand and a thumb. And I think that forms a lot of the basis for how to think about which gestures are comfortable and easy to use. To illustrate that, I have a video of me stalking a woman on an airplane. So here I am filming her without her knowing. I'd probably get arrested for this. But you can see how she's just using the thumb here. And this clearly looks like Facebook.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So she's going to flick, flick, flick. And then she tries to tap, tap. She does the two-hand thing. Tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap, tap. Screw it. It's back to flicking. I like that video because it illustrates, A, that I'm creepy, but B, It shows the behavior I was talking about there in the abstract form very realistically. It's that one hand, one thumb. You saw her put her additional hand up there so she could get to the higher edges of the screen.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And then the other thing you noticed is touch target sizes, pretty broken in that experience because they're probably just small link-based things in tiny text. and very difficult to hit. I think Craig talked about the scientific chart around gestures, and I have some stuff on that somewhere I could probably dig up. But in general, you want to make sure you have the right touch target sizes for hands and design for that accordingly. Now, what does it mean when we design for these kinds of use cases? Let me see if I've got a nice image. Probably do, but let's see here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And for reference, what I'm doing is scanning like 600 slides. So sometimes it takes me a second to find the appropriate thing. Let's open up another deck real quick and find it there. One of these will be it. Let's see. Not that one. I got a nice video of making use of those types of gestures and illustrating how they can work for you. So give me a secundo. See, I should have some jokes prepared for situations like this, right? Where I just sit there scrolling, scrolling, scrolling, awkwardly. And you say, what the hell is he doing? And I say, I don't know.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "All right, one more deck, and then I will just jump to this other piece. Screen, screen, screen. All right, if this isn't it, I give up, and I'll show you some other stuff. OK, I give up. I'll show you some other stuff. Actually, we can maybe talk about this. This is somewhat related. Let me show you this and then I'll switch over to the other thing. Some examples of how we can account for this kind of behavior.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "For example, here, this principle that I've talked about for a while, and excuse the dated example of the old style here, but there's A general principle that, and I think this got mentioned a few times yesterday, how typing is hard on mobile. I think Ani talked about this a bunch of times. There's a number of things that you can do to make typing easier and also take advantage of gestures for it. So here when you're creating this guide, there's default text up there, there's how to, and then there's these little buttony experiences where you can just tap install.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So how to install, and you're off to the races. So we generally have this principle of try not to let the keyboard come up. Is there other ways that people can do things without a keyboard? And Google Trips. has some nice examples for this. So when you are searching for a flight destination, for example, we're going to default the airport closest to you using geolocation. So you don't have to enter the airport closest to you. We're going to give you suggestions of what are the most popular destinations from where you're at. So from San Jose, people usually go to Las Vegas.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So again, you can just pick one of those things by default. And you don't have to type in your to destination. If you want to pick an airport here, again, rather than getting you to type, you can just hit that little location icon. It'll give you other airports near you. So if you want to change from San Jose to, say, San Francisco, you can do that without the keyboard coming up. So there you go. Now, if you want to enter a destination, you can tap that map icon. And again, you can scroll all the way around here and find a destination.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "These are kind of the most common places things go. And you can just tap like that. And it fills in that information, starts to tell you how much it's going to cost. And you're right back. And you're right back here. And now what we've done, you can see there's already results coming in. So if one of these suits your fancy, you're ready to go because it's sort of defaulted into a... March 8th through 12th thing, but you can tap on that date. And when you're in the date, here there's a nice big scrolling area, which again, you can kind of use with one thumb.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So you can tap on 8 to 12, or however you want, and contrast that experience. So we'll show it in video form here. So I tap on date, and I can just scroll through the months really quickly and easily. I can tap the 3rd, and then I can tap the 13th or 13th to the 1st. Tells me how much it costs. I tap done and I've already picked my dates through this nice fluid one thumb kind of UI. Contrast this to something where somebody ports over like a desktop pattern and this is kind of your standard desktop widget where you flip between the months, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And you can pick March 20th and then you tap again to get your return date and now you move through the months this way and you can go back here and tap a date there. In this case, we'll tap February 7th or February 20th. And now we're departing March 20th and returning February 21st. Anybody see a problem with that? But so this previous UI did a couple things. A, it took advantage of this one thumb kind of behavior, right? Scroll up and down, tap ones, tap the other. But it also prevented errors from happening here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so this is, I think, thinking more for this kind of UI as opposed to porting over your desktop UI, which I feel this is an example of. And hopefully this example illustrates kind of the contrast of, let me just take my website, shrink it to a small screen versus let me think about the reality of gestures. And there's other things that happen here. If you want to change price or time duration, you get a nice slider, which also is sort of optimized for those kinds of interactions. So you can flick it over here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And when you add all this stuff up, right, that's about seven different things that are being done in this small form UI to try to keep you off the keyboard, try to prevent you from typing. Use location, use the map, use that date picker, use sliders, all these kinds of interactions that are trying to help build for the reality that I illustrated earlier, which, again, looks something like And this is one of the things that I think we should always keep in mind, which is what is the reality of use? What are people actually doing?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Similar to those short bursty interactions, the number of times they interact. How are they actually holding the phone? How are thumbs sized? What can they do? What can't they do? All that can inform what we do with design and gestures. Sound good? OK, he gave me the thumbs up. All right, where else should we throw the microphone? I saw there's one behind you. Yep, two behind.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Yeah, could you talk about using drop-down inputs and not using drop-down inputs?", "start": 0, "speaker": "SPEAKER_09"}, {"end": 0, "text": "Can we talk about drop-downs and not drop-downs?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Especially on stuff that's not like when you can't use location, but a set of... We have to pick one of five options or one of 50 options and alternatives to using a drop-down.", "start": 0, "speaker": "SPEAKER_09"}, {"end": 0, "text": "What happened? Ta-da! OK, let's talk about dropdowns. Just so happens I have 50 slides on dropdowns. I don't know how that happened. I have this, so let me back up half a second. A lot of the ways I think about doing design is under this notion of a consideration, right? As opposed to hard and fast rules, like this is the way you have to do this, or this is the answer. Instead, I like to present things as considerations, which is stuff to think about as you're going through the design process.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "While you're building a mobile experience, what kinds of ideas or aspirations should you have? And one of these design considerations, aka something to think about while designing, is treating dropdowns as a last resort. And why do I say this? So let's look at, in the US, picking a state. So you tap the state thing. And if you live in Alabama, this is pretty much the only good thing that happens to you. Because you get to use these drop downs of just two taps.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "For everybody else, they have to scroll and find their state, and then that could be multiple taps, and then they have to tap done. So it's at least four taps in most cases. Kind of painful. And remember this example I showed earlier, sometimes drop-downs get used to an extreme. So in this form, departure date is three drop-downs. Return date is three drop-downs. Number of passengers is a drop-down. Cabin preference is a drop-down. Country of residence is a drop-down. Drop-down, drop-down, drop-down, drop-down.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And many people's design processes, this is the thing they reach for first. It's like their hammer, or their nail, or whatever. They always reach for the dropdown. But dropdowns, in many cases, are quite hard to navigate, as we saw earlier, especially on desktops. If you've ever gotten into a long dropdown, try precisely picking that thing out of the list. You can't really do any editing. It doesn't support hierarchy, and on and on. So I showed this pattern earlier where you type a zip code and you get back a city and state. In some cases, you may get back more than one.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But instead of presenting them as a dropdown, we can present them as a little stepper. And so a stepper would allow you to move through those options quickly and easily with a nice little touch gesture. And so a stepper is one alternative to a dropdown. And what a stepper will do is move things up or down. It'll sort of change a value through a consistent set of adjustments. And you can adjust the stepper to actually have an input value inside if you want. And there's lots of opportunities where these kinds of interfaces make a ton of sense.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So here, when you're picking a number of travelers in a booking form, you can just go, yeah, two. The most travelers you can have is eight. So in worst case scenario, you'll tap this thing seven times. Here's another example. Instead of using a dropdown, there's this multi-picker menu for the number of children, number of infants, which maybe it's easier just to do something like this. And if you look at Airbnb, they used to have guests and beds and bedrooms and bathrooms all done with dropdown menus.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And now they've moved over to these little stepper controls here inside of their Android app. Now, there's good and bad ways to use steppers. Just there is with every single UI element. So here on American Airlines, in their redesigned form, they do use a stepper. if you look at this thing, but I think there's a couple issues with this one, which is A, these things don't really look actionable because they're so light gray and off to the sides, right? It doesn't really feel like an actionable UI. And it's so separated from this.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And then if you go into this mode, if you will, let me just hit that, you enter like stepper mania. And again, if these things felt a little more like actual stepper controls, I think it would be better. But they kind of feel inactive, and it's unclear what's actually happening here. Now, another place that we saw dropdowns being used was sort of airport class of travel. So in this case, economy of restrictions. And if you tap on that, then you can see all of your options, including business class and higher. Business class and higher. First, premium infrastructure.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "By the way, do you think there's lawyers that work at American Airlines? I'm pretty sure a lawyer wrote that copy. Contrast that to what happens on British Airways, where they actually use this thing called English to describe There are travel classes, economy, premium economy, business. But this is a little action sheet. So this pops up from the bottom, and you can pick the type of class of travel you have. And an action sheet is another alternative here. This is kind of that bottom menu thing I showed you earlier. So you have a set of choices related to some task.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "It comes up from the bottom. You can have a cancel action. Works real nice. Looking at United, the way they ask class of travel is you tap on that and you come into this full page view. Which, though they're using English, this seems kind of excessive, right? But United, it turns out, is super into lists because even picking the number of passengers results in a list. So they love them some lists. Contrast the way all those folks ask for a class of travel to the way this site does it. I think this is Hipmunk, if I remember correctly.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And here they've reduced a class of travel down to a series of segmented controls, or in the Android world, radio button groups. So a radio button group is a set of related but mutually exclusive choices. You generally have five or less of them. And it's also a nice contender for this kind of stuff. Now, we looked at dates earlier, so I'll just sort of jump through dates really quickly. But in general, these kinds of desktop date pickers don't work nearly as well as something that's optimized more for mobile. And you're starting to see more people adapt to these kinds of controls.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Although, I don't know why Kayak makes it look like a real calendar. This looks like a printed calendar here. And I don't know why United inserts this big gap here. But increasingly, people are moving more towards single calendar things, which we'll see in a second. And once you start thinking about all the different places and ways you can replace dropdowns, there's a whole world of opportunity. You can use switches. You can use sliders. You can use segmented controls. Switches will let you do on-off type things, mutually exclusive choices.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Sliders will let you move through a range of stuff. There's good and bad ways to use each of these controls. For example, I wouldn't necessarily use a slider to pick the number of people in a hotel room. And you can style sliders however you want. So there's a whole slew of things available to you, right? steppers, switches, segmented controls, sliders, date pickers, dialogues. A lot of these controls are built for touch interfaces. And they're almost, you know, they're available in some ways on desktops, but a lot of them have been built with touch target sizes in mind.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And coming back to, let me just jump a couple slides to get to the point. Let me do it a faster way. Where are we? Here we are. At least I thought I was over there. So let's look at these things in action. So what's the point? Here's a bunch of different ways of doing what we did with dropdowns. We wanted to see, are these actually better than dropdowns in many cases? And so what we did is we took a bunch of apps for booking flight travel, and we specifically picked examples that use different controls to ask different questions.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And we compared which controls worked best for each question. Because what it turns out is these are your list of options, but different types of questions involve different types of answers. So different controls are better fit for different types of questions. To illustrate this, it turns out choosing travel dates on a single calendar was easier and quicker than having these two calendar things. So this pick-your-travel-dates UI that I showed you worked really well. For picking number of passengers, the most efficient and easy-to-use thing turned out to be the stepper.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This resonated really well with people. When it came to picking class of travel, the action sheet worked quite good. When you tap on it, up comes your options. But then also, the segmented control worked really well, too. It was almost like a tie there. And here, people kind of appreciated seeing all of the options front and center. And so, cumulatively, all these things, I think, really add up. If you remember the image I showed earlier, when you've got this drop-down written form, your first instinct is this is gonna suck.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Whereas if you have a form that uses the appropriate types of inputs for the kinds of questions you're asking, then you can really make something that feels like it'll be a lot easier to do, right? A lot more approachable. This is pretty easy. All my stuff is right here. I can jump in here and interact with it. Now, the question I always get asked about drop-down apps, about drop-downs, is, well, that's OK for easy stuff, right? It's easy to find a flight that puts you in a metal tin can that'll transport you anywhere in the world in hours.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But what about complicated things, like enterprise-y stuff? I think the same rules apply. In fact, this list of what makes drop-down menus hard to use I got from a company building enterprise apps. And what they illustrated is how they moved to a new series of controls to help people manage their invoicing system. And so they installed this system called the Picker. And this runs across their mobile, tablet, and desktop UIs. And if you hit the invoice due date, What you actually get is a little picker control that allows you to pick when things are due.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You get a bunch of default options, so you can do this quickly and easily. Or you can go and set a specific date, which, again, is quite hard to do with a dropdown menu. You can pick a specific date, and then it'll come back and fill that in. And because they built a responsive design, this works in kind of a layered fashion on small screens as well. Looking at another one of the elements inside of this form, if you want to change the units in the invoice, you don't want to have pieces. You want to have something else.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Not only can you pick between the defaults, but there is a search field at the top. So you can go in and type kilowatt. You can type or tap kilowatt per hour. There it is. And because they have these custom controls, next time you tap it, kilowatt hour sits at the top and the list gets updated. So it's an example of how alternatives to dropdowns can be applied to even things that are, quote unquote, more complex.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the reason why I harp on this stuff is, again, all these other mobile optimized controls can really help reduce input effort, make things feel more approachable, and basically get to the right question and answer pairing in terms of UI. And yes, there are situations where you might want to use a dropdown, but I prefer to treat it as the last resort. I can't do it with a stepper. I can't do it with this. Let me come back to the dropdown as my last choice. OK, I got another thumbs up. Yay. All right, where should the microphone go next? OK, it looks like in the middle here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Over there? Yeah? OK, cool. We'll make our way down gradually.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Luke, question on contextual design. Google's been working very hard on answering questions on the first page of the search results. How would you design something that I haven't asked yet?", "start": 0, "speaker": "SPEAKER_01"}, {"end": 0, "text": "How would I design something that you haven't asked yet?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I'm standing at the bus stop getting the phone out of my pocket. I want to see what time it takes until the bus arrives. Have you been working on that?", "start": 0, "speaker": "SPEAKER_01"}, {"end": 0, "text": "Yeah, how will I know what you're going to ask? I have this mind reading device implanted in my ear. I think there's a bunch of signals that we can use. Context, though, is a very challenging thing to infer. And so the question is, can we kind of infer what people think or what they're trying to do and maybe have preemptive interfaces? It's a very nice aspirational pattern. And I do have something somewhere on it. Yeah, I do. Let me bring this up and show you. It's inside this other deck. So give me half a second.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "The problem with inferring things is many times we are still wrong. We just don't have enough data to know necessarily what's happening. And you can see an example of this. If you go back to early days of mobile, did anybody ever end up on a mobile site that thought it knew you were a mobile user and only showed you the things that it thought were good for mobile users? Remember this? And I was like, here's the limited dumb dumb thing for you, poor mobile user. If only you had a large screen device, I could tell you everything about myself.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And so it kind of inferred that, oh, someone's on mobile. They're probably on the go. So we're only going to show them this and this info. The reality is people will use their mobile devices everywhere and anywhere. You're all using it now. Are you mobile? Or are you just farting around on Facebook? I saw a couple of guilty as charged faces. And so even though we know you're stationary and indoors and in an office, we don't necessarily know 100% your intent. So let me show you a way of dealing with that. And I think I have a nice series of images, if I can get to them.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "There we go. Now, this is the way I used to always illustrate what I was talking about before when this was an issue. Luckily, this isn't so much of an issue, because it used to be when we thought about taking our websites down to mobile, the question we had was, well, what do we get rid of from our website so we can make mobile websites? What sort of limited content and features will we get? And what you ended up with stuff is having Marriott, which has this big website about all of their different products and their destinations.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And then you had mobile version of Marriott, which was this simplified dumbed down kind of thing. And when you tried to get here, so here I'm going to tap getting here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Mm-hmm.", "start": 0, "speaker": "SPEAKER_09"}, {"end": 0, "text": "Or as it turns out, on the desktop version, they had this thing called Airport Shuttle Service that would actually tell you how to get to the hotel. But that wasn't on mobile. because they had cut that part out and what have you, even though it's actually technically useful information for mobile. So that's kind of an illustration of what used to happen. And hopefully, that doesn't really happen that much anymore. I haven't had to talk about that in a long time, which leads me to think maybe it's OK. But let me show you the other piece of the coin here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And I think I have a nice example. If I can find it. Almost there. Yes. Here we go. Okay. So here's a nice example of adjusting for context without denying people information. So if you look at this children's museum website, let me back it up and do it again. You can see that they have up front this big visit us, call us, and find us, and hours open information. There's still a menu in the upper right. And the menu in the upper right, if you tap that, will get you to all their information. You can go see the exhibits, education stuff, birthday parties.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Now when this same website is displayed on a larger screen, you can see contact us, find us, and the hours kind of get reduced in visual presence. They're still there, they haven't cut any information away, but they change the priority of information. So when you're on this smaller screen, chances are more likely that you're potentially trying to get there, right? And so therefore they prioritize that info, but they still have the ability for you to fart around on the couch and see the specials, look at their education programs, explore the content, and you kind of go this way.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So the question always comes up, well, don't people use these devices in different ways? And if you look at device usage, I've got a bunch of slides. These are sort of older ones, but I think the message is still the same. Desktop usage tends to spike during the middle of the day when people are at work. And again, they're farting around. So you'll pretty much see this pattern all the time for desktop. Tablets, in general, have kind of an evening spike. So like the morning spike and the evening spike happens on tablets, usually more of an evening spike in downtime.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This is now specific to Financial Times users, and you can see smartphone behavior happens throughout the day, but here there's a big spike. And for the financial times, maybe it's people on their commute. Maybe it's people checking in on the markets as they open. Maybe there's a flurry of activity right there. Maybe people are grabbing their phone and seeing what's happening in the morning. And then in the evening, jumping back on smartphones. This is BBC on the Olympics. You see the same desktop spike.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You see mobile's pretty big all the time, and then fills in the space on when desktop's gone. And you can see tablet again has this evening spike. And I think this quote kind of sums it up pretty nicely, which is PC and tablet usage, desktop and tablet usage have really big spikes, but mobile is sort of anywhere and everywhere. So it's always on in many ways. And while we try to think about context, when we think where do mobile searches happen, most of them don't happen when people are actually mobile. They happen at home and at work.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So while inferring context based on what kind of device and where people are can sometimes help us, many times we can actually be wrong. And frankly, I just don't believe right now we have enough information to really dramatically change experiences based on what we think somebody is doing or isn't doing. We're getting better, for sure. We're getting more signals. But it's not at that kind of, oh, yeah, that must be what this person is doing state. Cool? OK, where should it go? I'm going to throw it to the center there. Nice. I think that was nice. I couldn't tell.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I just heard the.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Yeah, I think he's fine.", "start": 0, "speaker": "SPEAKER_12"}, {"end": 0, "text": "Excellent.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So you encourage us to use this more natural way to, for example, log in or register. And generally use natural gestures in our products. But what if our customers use multiple devices? So how to make iOS, Android app, and desktop experience compatible.", "start": 0, "speaker": "SPEAKER_12"}, {"end": 0, "text": "Yeah, I think there's all kinds of really useful ways to start thinking about multiple devices. And this data also is a little older, but I think it'll... give you a sense of how much people do it. So in the US, half the people with a laptop also have a smartphone. A third of smartphone users also have a tablet. 13% have all of them. So there's a lot of cross-device kind of usage. And 90% of people that have More than one device will use devices sequentially. They'll go from the phone to the laptop, to the desktop, back to the laptop.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But then also, a huge amount of them do it not sequentially, but simultaneously. So here's the amount of people that use these devices simultaneously. While the TV's on, they're on their smartphone, or they've got their laptop. on their lap while the TV's on. And regardless of whether or not these numbers are still accurate today, the general pattern is still accurate. People will move between devices, although use devices at the same time. Which leads us to think about how can we design for cross-device usage, right? How can we think about experiences that take advantage of this?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And there's a couple patterns to think through here that may give you some inspiration. So one is this notion of access. And access is simply making sure that what you have is available across these different devices in the appropriate way. And the reason why this is becoming increasingly important is because people can transition between these devices much more seamlessly. So for example, if I'm on my laptop and I go to my website, I click on this link, I go click on this other link, I click on yet another link, and I just navigate it around this thing.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Then when I fire up my mobile phone, you can see here with Chrome, we're actually showing you that laptop where you were just at and how long ago we synced. So you can not only tap on that page and be exactly where you left off, but the back button actually works. And you can go and follow that whole path that you just took on your desktop on your phone. And so making sure that you have an experience that's available across these different devices makes these experiences work.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "If you have like a separate mobile site or a separate tablet site, the URLs don't align, this kind of stuff can break. Another example is flow. So content, that's probably not the only thing we want to move seamlessly between our devices. Processes should be able to move pretty seamlessly as well. So with this, if you're in the middle of a task, you can just pick up right where you left off when switching tasks. Again, I'm on my laptop. I search for a destination so I know where I'm traveling.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "When I open up my Maps app, in this case Google Maps, and I tap search, there's that location I just looked for and I can take off running. So you can just pick up your flow right where you left off. Control is another interesting pattern where one device controls another, often remotely. And so here we built a, oh, actually I thought I had a video of this. Is this the video? Let's see, maybe I just skipped through it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So we built an experience with Microsoft a while ago where you can sort of talk to Xbox and you can connect your phone to a big screen and you can see I can actually be voting on things. as they're displayed up on the TV. And the TV interface has a very different UI than what I have on the desktop. And I can watch TV in the one screen. And on the other screen, I can see while I'm voting here, I can go and do a search for Futurama. And now all the Futurama questions pop up there, and I can interact. But you get the idea here, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "One device is controlling either the other device or part of the other device. And the last one is a push, and this is the one I think is actually most interesting and perhaps underutilized. So eBay has a nice example of this. If I'm on my laptop and I'm listing an item for sale, I can say I'm going to grab an image from my phone. A little notification pops up on my phone. It opens the eBay app and jumps right to the camera roll. And so I can hit the camera roll, and I can use my phone, which is always in my pocket, to take a picture of the thing I'm selling.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So I just hit that, take a picture. It gets saved. I hit Done. And then automatically, from having taken that picture, it will now instantly show up on the desktop without me doing anything. So it's a really nice example of these devices working together. This has a camera that's always likely with me. Getting pictures onto the laptop is kind of hard. It's sort of a pain in the butt. So having this seamless integration through a push notification, it gives me an opportunity to use those devices together. So hopefully that gives you some ideas on how we can think cross-device.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Again, the general patterns are make sure your stuff's accessible. Responsive design is a great tool for that on the web. We can talk a little bit about how to make a consistent experience across native platforms as well. Flow is this notion of processes. Control gives you a chance to have one device control the other. That's potentially more rare. But this push scenario, I think, is the most exciting one, where you can use the best part of each device together, cumulatively, to create a overall experience that works well. OK. Answer your question? OK. Three thumbs up today.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I'm doing awesome. Where should we throw it? Looks like right behind you. Easy throw, yes.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Hi. Hi. So you talked a lot about the checkout process, how to make it simpler and smoother. Do you have any tips for what happens before the checkout process? So for example, making it simpler to choose from products or boosting the motivation to buy and so on?", "start": 0, "speaker": "SPEAKER_07"}, {"end": 0, "text": "Yeah, so in terms of like search patterns and the like? There's a whole series of things we can discuss around that. I'm trying to think what's the fastest and easiest way we can talk about it. Let me scan for half a second. I'll look at one thing here and see if it helps us. Oops, that's the whole talk I showed this morning. We don't want to reference that again. OK, so here we go. One more deck open recent. OK. So I think there's a lot of opportunities that we have to improve overall browsing experiences in a number of different ways.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Right now, we're kind of dependent on search and browse as our primary mechanisms. But for instance, whenever you go and represent any kind of like hierarchical list for people to find things, there's an opportunity to actually get them invested in what's inside of each category. to help them make their decisions appropriately. So for example, this doesn't necessarily apply to shopping, because this is in the finance space. But contrast that browse path on the left with this one on the right. You'll have a sense of what's inside. You have a sense of some of the activity.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And you know why you should access any of them. And then here, this is sort of the difference between kind of a static browse a category list and what you can do to make that a lot more dynamic and engaging. So here in the books department, not only is there search within a category, there's actually merchandising that drives you to popular stuff. There's this primary nav, this sub-nav. And so introducing these kinds of elements to any kind of guided browsing path can give people clues on where they should or shouldn't go or what they could do.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So I think those are a couple things to do for sure. In terms of increasing motivation on view item pages, I feel like you guys are doing enough already. It's just me. There's like 15 million things that tell me I should buy the darn thing. So I don't necessarily have too much on that. Next. OK, you've got to really throw it down. There's like four down here. Ready? Incoming. Whoa! Oh! That was almost perfect.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So actually, It is related to the last question. We are really struggling on our webshop with navigation. We have this old-fashioned hamburger menu. I thought about it when you showed the slides about the reach of the thumb. So a lot of web shops and also our web shop still have the hamburger on top, far away from the thumb actually, which is quite weird to my opinion. But I'm wondering about, I'm really curious about what opinions you have about optimal navigation on web shops that sell a lot of categories and if we should move it to down or", "start": 0, "speaker": "SPEAKER_10"}, {"end": 0, "text": "Other ideas? OK. So let's talk about that for half a second. So the hamburger menu pattern, for those of you that don't know, it's kind of you hit a little element in the upper left, and you go to the menu on the side. So here you can go and hit the hamburger menu. You see some navigation. You hit it back. It goes away. It looks like that, and here's another example. This one over on ESPN, you hit it, it comes back, goes away. And this used to be the case on Facebook a long time ago, which is up here.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "In fact, the hamburger menu has become so popular, it has its own Twitter account called Mr. Liney. And he basically goes around and tweeting all the places that he gets used because it gives him a nice ego boost. Now, there's good reasons for him to be happy about this. If you go and look around, you see hamburger menus everywhere. They're all over the place. And when you talk to people about why are they using hamburger menus, you generally hear people say something like, oh, everybody's doing it, so I will too. Or you hear Facebook is using it, so I will too.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And in particular, the second one, I think, is very, very dangerous because you don't know why they're doing it. I for years have gone and talked at companies where I've been really terrified going in there because I feel like they know everything. Oh my God, it's eBay. They know so much about mobile commerce. And when you come in, you realize they're figuring stuff out just like the rest of us.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And in particular, the case of Facebook, though they had this hamburger menu, When they did a redesign, they found that engagement, satisfaction, revenue, perception of speed, everything they cared about, actually did a lot better when they moved to this. And the reason why they ended up sticking with that hamburger menu for so long is because their app was really damn slow and buggy and it used memory. It was essentially JavaScript creating a pseudo web browser inside of a native app inside of the OS.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So they had a one or two star review for a long time because it was so crappy and slow. So they were spending all their time rewriting the app and not thinking about the nav. When they implemented an A-B testing system and they could compare the nav against the other, they realized, hey, this actually works a lot better. So be careful when you look at what other people are doing and assume reasons why it is that way. It could be God knows what reason. You know why the hamburger menu is on that person's site? Because Bob the developer and Jane the PM hate each other.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And Bob refuses to implement the new now. And Jane escalated it to her boss, and Bob's being such an a-hole. You don't know. Human beings are human beings. There could be a gazillion reasons. So it's always best to go and get your own opinion on things. And I went and talked to a bunch of people around the hamburger menu. Here, some people got it, thought it was an Apple thing. Poor Danny at AT&T found people have no idea what they are. That's a shame. And then you can learn more through these qualitative things, talking to more people. You can run quantitative tests.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "James Foster ran this quant test where he took the hamburger menu, compared it against the hamburger of the word menu, compared it against the hamburger inside of a button type thing, and found adding the word menu helps people understand it. But making it look like something you can actually tap helps more people understand it. Shocking, I know. Then he took the winner of round one, and he tested it against the Word menu. He tested it against the Word menu with an icon, and he tested it with just the Word menu.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And what he found was using words instead of cryptic symbols tends to improve things. Shocking, I know. He also found that having the icon in the menu doesn't do that much. And this one is the most funny to me, because it turns out like 22% of people need a buttony thing to understand something's tappable. Because before, remember when you put the buttony thing, 22% more used it. When you took away the buttony thing, those poor 22% of people were like, oh my god. So then he went and ran a grand finale.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "He took the hamburger menu in the buttony thing, the word menu in the buttony thing, and he ran it against almost 250,000 unique mobile visitors. And it turned out the word menu outperformed. which is cool. Now, of course, all of this matters your specific context. And I'm going to reference Booking.com because they brought this up. And I think somebody, maybe it was Craig, quoted from them. When they ran this test, they found no significant impact between hamburger menu and the word menu. But it turns out, what is behind the hamburger menu on Booking.com?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "The ability to change the currency in which your results apply. your ability to create a profile. In other words, this is not critical stuff. And so in their specific user base, it may or may not matter that people even access that menu. How often are you changing your currency settings? Hm, doesn't matter so much. In other situations, and by the way, I think if they had run this test, it'd be different. But anyway, in other situations, it matters a ton. So let me skip this slide and look at this slide.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "For a time, what they did when they moved to the hamburger menus, they took all the categories of their newspaper, world, politics, entertainment, sports, they hid them behind this tiny little thing. And guess what happened? To their metrics. Because people didn't know where the sections were anymore. Where's sports? Where's news? Where's entertainment? And they got so desperate, they had to add the word menu. And then they added this thing. New time, new navigation. Click the menu. OK, OK. Desperation, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You don't need the qualitative studies and the quantitative studies that I showed you a second ago sometimes. You just need to see something like this, these Band-Aids. Look for the Band-Aids, and that's where you'll find the ouchies, the usability ouchies. And it's a nice illustration of we shouldn't just copy patterns, right? To come back to this for half a second, where did I put this? Because copying patterns, again, we don't know why things are the way they are. So if we come back to this, this was hamburger menu, post-hamburger menu for TabBar on Facebook.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And you can see the before and after view. And they had better engagement, satisfaction, what have you. Another example, in this app redesign, they used to have a hamburger menu, they moved to the tab bar, session time increased 70%, 65% increase in daily active users. And I should pause because this is super dramatic, right? How is this even possible? Well, what happens is, before, all of this is out of sight, out of mind. And here, we're making it not only visible, but we're showing activity, telling you what's actually happening in this app, putting it front and center.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So you know there's things that are meaningful to you. Sort of looking at it in the opposite direction, this app went from a tab design with a set of options around what's inside of this entertainment app to this navigation drawer pattern where they hid everything behind their icon and you have to slide it open to see it. And when they did that and they looked at their analytics, they said it was a disaster. Weekly and daily frequency was down. In fact, weekly and Engagement time was halved. Time spent in the app went down.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So they lost half of their activity when they buried these different features and functions, which is a huge amount. So what does all of this stuff mean? I think what it means is we want to make visible to people what is inside of our apps, why they should use them, how they should use them, and those kinds of things. And last year, I won't repeat it this year because we don't have enough time, but last year at this conference, I walked through a whole redesign going from a hidden nav structure to a visible nav structure. And I outlined what the big benefits of that were.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "But what it basically boils down to is very, very similar to what you see inside this image here. And I actually think I've got even more examples of this now. It turns out examples of this keep popping up left and right. Let me see if I can find this last one. And so my advice around navigation is, first and foremost, understand the things that help people orient to what you do. Here's the other one. Let me play that.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So here's another example where they went from this hamburger menu down to this bottom navigation menu, double-digit growth in monthly active users, double-digit growth in visitors, triple-digit growth in users, what have you. And so it's just front and center what the main pieces of the application are, what you can do there, and what you should do there, whereas before, everything is often hidden. And I guess maybe I can dig up one chart of what I showed last year to tempt you to go look at that video, if you so choose. Let me find it. Here we go.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So when we put some things inside of the bottom navigation menu, here's kind of the chart we saw. This little chart is the test of it. And then that's the actual one of it. And I have similar charts for all kinds of other important-y things. Does that help answer that question? So this is, again, I think comes down to the trick of designing for mobile, finding the right stuff to put on the small screen at the right time.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "When people enter a service or when they're at the top level, one of the things that I think is the right thing to show them is the scope of what's possible and the means to get to that possibility. And so navigational systems, which hide all of that possibility and get rid of it, do a disservice to people. They don't know what's available. Therefore, they don't get into it. And as a result, you may end up losing them from an engagement perspective. All right, I don't know where the mic is anymore. Who's next? Let's go up. There you go. Nice.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Hi. In the previous session, you gave the example of a newly designed credit card form, which on first glance looked are pretty complex to handle. So can you give a little more detail to that?", "start": 0, "speaker": "SPEAKER_08"}, {"end": 0, "text": "Sure, on that credit card pattern thing? Yeah, I actually have a PDF where we tested it and show how it works. But let me see if I have a little more detail about the actual interaction design of it here. Wrong deck. Let's see. I want this one. No, I want this one. Yeah.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So here we go.", "start": 0, "speaker": "SPEAKER_06"}, {"end": 0, "text": "I got a video of it, so this should help you. So this is an implementation of it on the web. And so effectively what we do is we bring up, when you tap into the field, we bring up a numerical dial pad. So it's all just numbers. You can't type anything but numbers. And as you type in, you'll note if you enter an incorrect credit card number, it'll throw an arrow and not allow you to continue. If you put a appropriate credit card, it'll identify that credit card, switch it to the type you're entering, so it lets you know it's a MasterCard. You don't have to set credit card type.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Then once you get to a valid credit card number, we just slide it over. And then you're on to the next field, where you're entering the expiration date. And again, it's just numbers. Now we show you where your CVV is. Again, it's just numbers. And this specific example also includes a zip code. So if you want to capture that, you can. You don't have to. But all of this information just gets collected right here on these numbers. And you get this real-time feedback, what have you.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "When we tested this with people, to give you some of the detail of what we found, number one, people were very happy to have this big numeric keypad. They responded really well to this because they didn't have to jump between keyboards and figure out what's going on. They wondered sometimes how to go back to editing their credit card number. Not that they needed to, but they brought it up. And so effectively, the ways to do that is to either tap on the number or to tap back. And both of those things work, so yay.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the one thing that they were actually pretty discontent about, we tested a couple implementations of this, including the one Square uses, where they put dots on there, and people hated the dots. Because it masked what they entered. Does that answer? Yeah, so maybe it feels a little complicated walking through it this way, but it's just like if you look at your credit card. I think I have a picture. If you look at a credit card, you're literally just da-da-da-da-da, da-da-da-da-da, flip it over, da-da-da-da-da.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And it's all just numbers, and you stay on that input the whole time and put it in there. OK, we're to. This man's been patient.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Thank you. I would like to talk about ethical way or like philosophical way how we design actually. Like I remember from the last conversion of your presentation last year that like spending time on mobile was increasing. And then today, you also mentioned that sessions in mobile also getting more often and frequently in the daytime. And to build more habit-forming products in the industry now, we are working with people from different fields, like cognitive science, psychologists, sociologists, and so on.", "start": 0, "speaker": "SPEAKER_03"}, {"end": 0, "text": "And what we do as we design is that we capture the user, their micro-movements to get their attention. to show, to sell, and so on. I just want to ask, do you think building better product necessarily means that we are building a better life for the people, for the users? Because they could also use that micro moments time spending, I don't know, having a dinner, or having chat with friends, and so on. Do you think necessarily spending more time means a good thing for humans?", "start": 0, "speaker": "SPEAKER_03"}, {"end": 0, "text": "I'm biased, but I personally think it's awesome that interactive devices are taking time away from passive devices. Because if you look here, this is in the US, how many hours of TV people spend? Four hours a day, on average, staring at a TV. Is this good for humanity? We're going to have an ethical conversation about that. Maybe that's good. Maybe that's bad. But my small biased opinion, at least as someone who has children, I much strongly prefer if my child is building something in Minecraft than if they are staring at a video. That's just sort of a one way conversation.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And that's just my personal bias. Other people may have different opinions. So that's one kind of thing. And then the other piece is, what kinds of products are we making? I think there's a lot of products that are empowering, that give you superpowers, that allow you to do stuff you couldn't ever do before. And so I don't view those things as a detriment of attention. I view those things as an augmentation of attention. By using this, you can now navigate the entire planet. You can learn a foreign language. I mean, there's all sorts of things that help our lives.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "There's certainly things which are distraction for distraction's sake to sell attention. That's called advertising and that's existed for a long, long time. And it's not unique to mobile, right? People's attention is valuable and therefore you can monetize it. So I think it boils down to what are you making and what kind of perspective you take on it. My personal perspective is this increase in time spent away from these passive forms of screen sucking are a good thing. And we hear all the time, people just keep staring at screens. Yeah, they do. Sure as hell do, right?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And it's not the one you're talking about. It's not the little powered computer that's in your pocket. OK, behind you.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Hello. Hi. Do you have any examples on? what works for mobile around the filter design? Around which kind? Filtering search results.", "start": 0, "speaker": "SPEAKER_06"}, {"end": 0, "text": "Yeah. Oh, you know what? I'm probably not going to have it with me. But there is a book by a woman named Rachel Hinman. where she has, I think it's called the Mobile Frontier, and she has a series of patterns around filtering. I used to have a bunch of this stuff in my deck, but let me see if I do. For some reason, I feel like I don't have it here. I'll look real quick. But she has a series of patterns that you can use for filtering behaviors on mobile. Looking real fast. Yeah, I don't think I have it in here. But if you look up Rachel's book, Rachel Hinman, you'll see it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "She's got a whole chapter on all of those, hopefully useful. OK, onward. OK, this guy's got a finger pointing at him and his hand up, so win.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Thank you for the great stuff today. I was wondering what technology will replace smartphones, and when will it happen?", "start": 0, "speaker": "SPEAKER_04"}, {"end": 0, "text": "Oh, awesome question. So for years, I've gotten this question around, what's next after mobile? Is it VR? Is it AR? Is it wearables? What's it going to be? And for years, I didn't have an answer to this question. And now I feel like my thoughts around this are coalescing a little bit from being around people smarter than me. And the things that I think are coalescing are our ability to take huge amounts of data and learn things from them.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So if you look at some of the recent advances, and I know there's a lot of hype around this, but I've personally had several instances where I have been super impressed by what we've been able to do. And I'll try and find one of these things. But we, meaning Google, and the hype, meaning things like machine learning, So just very recently, we announced a couple of things that illustrate just how many parts of our day-to-day lives this stuff will impact. And I don't think I have a screen for this, sadly, so I'll just illustrate it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "One thing that we did is, remember those terrible little CAPTCHA boxes that were all over the internet that you proved to be a human? We now, through mining huge amounts of data, can just know whether you're a human or a robot. CAPTCHA is dead. This is the single biggest contribution of machine learning to date, in my opinion. The other thing that we just launched, we launched a program or a system called Razor, which what this does is it, on your device, upscales images using an algorithm.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So we send images that are 75% less bandwidth to your phone, and your phone makes them look awesome. through by looking at billions of different images. We write the way to upscale them using an algorithm to make them look awesome on your phone. So again, think about the impact of that. If we can cut down on 75% of bandwidth of images across mobile, across these 5 billion devices, look at the impact that kind of thing has. YouTube recently announced they had projections for 400 million viewing hours. per day or something like this, and they crossed a billion.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And the way they did that is, again, through this deep learning of huge data sets where their recommendations just got awesome and they gave you content that you actually cared about. And so they more than doubled their goal. And so just with those three kinds of examples, you can sort of see how this will affect all kinds of things. And the stuff that I'm particularly passionate about, because you can hear me ranting about checkout and sign-in. Imagine how I feel about CAPTCHA. Hate CAPTCHA, right? Got rid of CAPTCHA. You don't have to do anything.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And it's a perfect example of what I was talking about earlier, where we take all that complexity and do all this stuff just so you can have a normal, awesome life. So for me, that's a really big trend that is super exciting. And if you look at, again, the advances in translation, the advances in image recognition, this stuff is transformational and very exciting. And then the other trend that I think is also very exciting is cue the screen. We used to just make software for this. Now our canvas looks a lot more like this. And what kind of implications does that have?", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "What sort of changes the notion of what is a personal computer? if you will. There are things in the home which can collect data, and the things in the home which can collect data can have output on a variety of screens. There's things which can be on you that collect data, and that data can have output on a variety of screens. And so when you start to combine those things And you start to combine other kinds of data input things, and you look at things that could be public.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "I think when we build this really quickly, I feel like the next definition of a personal computer is actually this mesh of endpoints, if you will. So there's things which out there are focused on input, collecting data. And then there's things that are out there which are focused on output. which are displaying those things. And instead of having input and output just in one device only, it's more of an integrated world. And your personal computer is a combination of your interactions in the car, things that are collecting data about you, stuff inside your house.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "This world is almost here yet. It's just not cohesive yet. And the other nice thing about this, again, if we project forward a little bit, is, in an ideal world, you just sort of pick up any screen, and it is kind of like a dumb terminal to whatever data source or service you want. Now you can just fluidly switch between screens. Oh, let me throw that up on the wall. Let me push this to my laptop, what have you. I think a lot of these things exist in micro situations, but cumulatively we're not yet at the state where this stuff is fluid, although it feels like we're getting there.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "So if you take this kind of environment. And then you take our ability to make sense of all this data that we're collecting and use it to help you with your life and the examples that I showed earlier and the many more that will come. It's going to be pretty damn exciting. Yeah. I think this might be the last one. No pressure. We've got to get you to lunch.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You showed us the numbers on Cyber Monday versus the Singles Days in China. Do we have a clue what they are doing different to get these numbers on the mobile device preference?", "start": 0, "speaker": "SPEAKER_02"}, {"end": 0, "text": "Yeah, so what's the difference between Cyber Monday in the US and Singles Day in China? In China, they're thinking mobile first. I mean, I think this is the baseline difference, right? They don't have necessarily this wide distributed PC base, nor do they have this mindset of, oh, desktop, right? People convert on desktop. They're just all in on mobile all the way. And so the deals they offer are mobile-centric deals. The experience on mobile for buying has been built from day one that mobile payments are easy and frictionless in their app.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "And again, there are examples of services that are At these higher bars, the Walmarts, Targets, the Amazons, they've made investments in mobile. They run campaigns on mobile, they optimize for mobile. And it's the combination of those things together that I think really boosts that number. In particular, Alibaba, I don't know, 110% if they did it this year. But in the past, they have run mobile-only promotions. So you only get the deals on mobile.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "You have to go to mobile to get it, which is a great way to drive their mobile usage up, get people used to buying on mobile, and the like. And while I think, oops, while I think, you know, we can look at these different countries, I'm going to just bring up one thing here. Oops, wrong deck. Let me switch over to this. If we look at what's happening in even the U.S., here's my nice chart. Can we show that slide real quick? Which screen is it? Do you see it? No, not that one? OK, I'll move it over. I can't see my mouse. Where is it? Oh, now you can see it.", "start": 0, "speaker": "SPEAKER_05"}, {"end": 0, "text": "Sorry, third monitor over. If you look even in the United States, the number of mobile-only users is growing. So now nearly one in eight people on the internet in the US are mobile. only, whereas the percentage much higher in other countries and regions. But even in developed regions, this sort of behavior is popping up. OK, cool. Hope some of this was useful. Thanks for asking great questions. If you want to talk more, I'll be here for another couple of minutes. I believe we need to wrap up now so you know which buses to take and you can get to your lunch. But thank you very much.", "start": 0, "speaker": "SPEAKER_05"}]}]},"created_at":"2025-03-27T23:59:22.466274+00:00","method":"reply","model":"claude-3-5-sonnet-20240620","additional_info":null,"context_retrieved":["9e728f85-426f-4055-bf2a-7434ec08a366","b58c796d-162a-4882-94d6-2f63da89d016","2a247159-cf3d-4e22-8297-0425f8cf87d7","210ef1aa-5256-4cd2-bb2e-ae93d1f8d986","3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a","e2392088-1a25-4c60-84fb-d50972b7bc9b","77e1b99b-7135-40ed-a347-e69985872667","21b08c51-bb49-4ee5-989e-72aba8d20948","0633eac7-8853-46ca-b8be-5720b129c2cd","c3e70d50-9feb-4c99-b822-62e6d1c32c45"],"context_all":[{"id": "9e728f85-426f-4055-bf2a-7434ec08a366", "locked": false, "source": "EMBEDDING_CHUNKS_VIDEO", "similarity": 0.6400000000000001}, {"id": "b58c796d-162a-4882-94d6-2f63da89d016", "locked": false, "source": "EMBEDDING_CHUNKS_SAVED_QUESTION", "similarity": 0.6400000000000001}, {"id": "2a247159-cf3d-4e22-8297-0425f8cf87d7", "locked": false, "source": "EMBEDDING_CHUNKS_VIDEO", "similarity": 0.6400000000000001}, {"id": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "locked": false, "source": "EMBEDDING_CHUNKS_LUKEW", "similarity": 0.6400000000000001}, {"id": "3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a", "locked": false, "source": "EMBEDDING_CHUNKS_LUKEW", "similarity": 0.5599999999999999}, {"id": "e2392088-1a25-4c60-84fb-d50972b7bc9b", "locked": false, "source": "EMBEDDING_CHUNKS_SAVED_QUESTION", "similarity": 0.5599999999999999}, {"id": "77e1b99b-7135-40ed-a347-e69985872667", "locked": false, "source": "EMBEDDING_CHUNKS_VIDEO", "similarity": 0.5599999999999999}, {"id": "21b08c51-bb49-4ee5-989e-72aba8d20948", "locked": false, "source": "EMBEDDING_CHUNKS_LUKEW", "similarity": 0.5599999999999999}, {"id": "0633eac7-8853-46ca-b8be-5720b129c2cd", "locked": false, "source": "EMBEDDING_CHUNKS_LUKEW", "similarity": 0.5599999999999999}, {"id": "c3e70d50-9feb-4c99-b822-62e6d1c32c45", "locked": false, "source": "EMBEDDING_CHUNKS_VIDEO", "similarity": 0.5599999999999999}],"context_diversify":["3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a","ff74d55f-8fd9-4bfd-9897-b26404d7aa1b","512c9ed2-93c9-4ed2-92c4-b2749c18c3d6","1d2d8342-f27d-49a5-bfba-88d5f0b3d909","e2392088-1a25-4c60-84fb-d50972b7bc9b","77e1b99b-7135-40ed-a347-e69985872667","e2af7877-9555-42f6-99c9-a9988d8b4d23","9e728f85-426f-4055-bf2a-7434ec08a366","b58c796d-162a-4882-94d6-2f63da89d016","21b08c51-bb49-4ee5-989e-72aba8d20948","15982328-db57-45c9-8c86-c95a454c9f32","4d717464-efb8-411f-94f2-0cb2ab89bd1f","0633eac7-8853-46ca-b8be-5720b129c2cd","d19588f6-98ab-40dd-aac7-8db9630f0ce0","216c2710-5ad4-4133-b115-e1506e0e3151","70444fa3-dced-43e9-af71-f9ba040a3ae8","c3e70d50-9feb-4c99-b822-62e6d1c32c45","f4eb67a4-2f91-4529-8f52-c4cc731db0f4","2a247159-cf3d-4e22-8297-0425f8cf87d7","dddad87c-14a8-4157-9429-8996deb858ba","90770e86-b2e9-4082-a3b3-3f7f027461c3","aa69b3c6-6da9-46ea-8db3-1b3fc30bf15f","5353c0b4-d4b5-41f8-bb56-3a44e6dbef8a","17180b32-55bd-425e-9e66-dc76208a6b6c","2a02474c-0723-4d1e-8b93-4ebefa9ea207","10e73f58-5c9d-46cd-b98c-51d5b3aa8c03","8d873236-64fc-4142-8277-c1d1151f1026","17b5ad42-5a24-41d7-a2d3-05ae55be87d7","96fe63a3-c138-4993-a6d7-b7fdea674991","ee35527a-df2a-4d3f-9753-992d91b1e0ed","2326f59a-09bb-4ce5-9e9c-83fcf17fa76e","041c8efe-5a94-40d6-a5bf-60cf4a650c01","27851221-06aa-439e-a6dc-2440a54aac44","dd1151ab-b2c5-40e1-8ab5-63ca6be5d825","210ef1aa-5256-4cd2-bb2e-ae93d1f8d986","f5b5508a-e4ad-4d5a-bd42-f3537304d43f","88d83b0d-f05d-492b-8886-50c95d24399b","39ff196d-7691-48d2-a7d1-6a69ae1c36e3","836f9aae-4ed4-461c-82f0-47bfdf907860","ec9f8676-890b-4e9d-86ec-0ce7ecb76b0c","0fe9a800-4081-41f1-b338-5d8f7beda9eb","c4764b62-fb7d-4938-97ee-c37426134a89","285ef615-19f0-4f2b-8129-d7c9e6b98b67","e8fec4d5-d181-4f55-8416-325095c1a384"],"context_diversify_removal_details":{"411eac2f-6a65-4d20-86ad-c5afea4dcdf4": {"detail": "2326f59a-09bb-4ce5-9e9c-83fcf17fa76e", "reason": "similarity_diversity"}, "4700c97e-e2ca-4089-9cfa-14b8a625b4a6": {"detail": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "reason": "similarity_diversity"}, "6f8e761d-46bd-4c64-ad6e-14c44c609f68": {"detail": "39ff196d-7691-48d2-a7d1-6a69ae1c36e3", "reason": "similarity_diversity"}, "76233c74-b70e-4e5c-bbbc-5fa541c5c970": {"detail": "17b5ad42-5a24-41d7-a2d3-05ae55be87d7", "reason": "similarity_diversity"}, "7de32ee8-0637-4f53-a8c3-61982e3c0e6d": {"detail": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "reason": "similarity_diversity"}, "8194f526-8fd8-4602-901f-bd792d3a1434": {"detail": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "reason": "similarity_diversity"}, "afcb9130-1af8-42c7-a310-a192e872b34e": {"detail": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "reason": "similarity_diversity"}, "ce60eaac-01e2-4682-b5d3-e6a046271582": {"detail": "210ef1aa-5256-4cd2-bb2e-ae93d1f8d986", "reason": "similarity_diversity"}},"context_in_prompt":["9e728f85-426f-4055-bf2a-7434ec08a366","2a247159-cf3d-4e22-8297-0425f8cf87d7","210ef1aa-5256-4cd2-bb2e-ae93d1f8d986","3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a","77e1b99b-7135-40ed-a347-e69985872667","21b08c51-bb49-4ee5-989e-72aba8d20948"],"context_replied":["3cdb0e74-06d1-4bb3-8474-2eb2e65fe42a","2a247159-cf3d-4e22-8297-0425f8cf87d7","9e728f85-426f-4055-bf2a-7434ec08a366"],"question_keywords":"future publishing AI\n"}]

Here's some questions you can ask Luke Wroblewski