You Can Build an App in 60 Minutes with ChatGPT - Ep. 5 with Geoffrey Litt

[00:00] Why don't we just build something together? - All right, let's see how it goes. So I'm gonna kick us off with my goal, which is I'm appearing as a guest on a podcast. I want an app that I can have open while I'm on the show to help me keep track of time and remember what I wanna say. Okay, so we got this code. I'm not gonna read it. I just copy paste and go. I'm just pasting from ChatGPT. This should update live this little window here. - Oh my God. - Okay, look at that. [00:25] Thank you. [00:36] Jeffrey, welcome to the show. Thanks so much. It's good to be back. Yeah, good to have you. I interviewed you, I think, like two or three years ago on Super Organizers. That interview went really, really well. And I just think you're one of the deepest thinkers about this stuff. And in the pre-production call, I was just saying, like, I think generative AI just sort of like, it just takes the things that you've been thinking about to a new level. And I'm just really excited to check in with you. Yes, a lot has changed over the past couple of years, indeed. [01:06] And for folks that are not familiar with your work, you are a researcher at Ink and Switch, which is an independent research lab, and you explore what you call malleable software, which are computing environments where anyone can mold their own tools to their needs. Can you talk to us about what malleable software is and why that's important? [01:26] Yeah, absolutely. So this has been a quest that I've been on for at least the last five years now. And it all started, I was working at a startup and we were making SaaS software for schools. And

[01:38] I just started getting really bummed that we kept saying no to every single request that came our way from some teacher in a classroom. And they would say, you know, can you change this word because it doesn't match the way I teach? Or can you add this little button for me that would do X? And we would say, no, you know, we're not going to add that button just for you. And that's such a normal way of thinking about software for people used to SaaS. But it felt really strange to me. [02:08] have unique to themselves and not just using these pre-made tools. That's what malleable software means to me, is that we can all have the software that works for us, not for some product manager off in San Francisco. And the really interesting thing that's happened the past couple of years is that until recently, [02:26] Programming was a huge bottleneck in this space. You need to do a lot of programming to make any sort of software. And the big question was, how can we sort of route around programming to give people tools to make software without doing that? [02:39] And what really excites me about generative AI is that [02:43] oh my gosh, we now have a thing that can kind of sometimes turn fuzzy ideas into code. And so that's starting to cut almost like a shortcut through that impassable mountain. And I think it's a really exciting time to be exploring how can we apply that base technology to help everybody make and edit the software that they want to have. [03:05] Right. I love that. It's an idea I've been thinking about as well. You're obviously far deeper than I am, but I wrote this article called What Comes After SaaS, I don't know, probably like six months ago. And the short answer is like bespoke apps customized by AI.

[03:35] you know, when you use Spotify or Gmail or whatever, it's sort of like staying at a hotel where the company, like you, everyone gets the same hotel room. And, you know, the company is running the hotel. And when you sign up for, for Gmail or whatever, you get your little Gmail hotel room and it's got all the same things as everybody else. And there's like some ways that you can customize it. Like, obviously like you put your luggage in the hotel room, like, so there's certain things that are, that are yours, but yeah. [04:01] Everything is basically the same basic model. And over time, companies, especially SaaS companies, get good at adding little customizations or whatever, but it's always on their terms. It's always on their guardrails. [04:31] having their own [04:32] general contractor on staff that they can deploy to make any change they want. [04:38] Yeah, I love that analogy. I think it's a really good one, too, because it's subtle in that sometimes hotels are fine. And there's a whole spectrum from renting a hotel room to renting an apartment to buying a house that someone else built. There are people also who build their own home from scratch and design it all. Right. And there's a whole spectrum there. And it's okay to, I think, at different times be on different points.

[05:08] What would be a better layout? When I walk into a hotel room, I don't start thinking about moving the walls around or changing the decorations because it's not a thing that I obviously am allowed to do. And so those questions don't occur to me. And that's what I worry about with software is... [05:23] The worst case outcome is that we all use SaaS tools that subtly affect the way we think, and we don't even realize it because we're just so used to living in those tools. [05:32] Yeah. Yeah. I was just talking to a couple of weeks ago, I was talking to Linus Lee on this show and he was talking about something similar, which is the idea of agency and how our tools shape the agency that we have and thinking about ways to give users more agency in the context of general AI, where I think, you know, in large part for now, there's maybe a little bit less agency in something like using MidJourney than you might get in something like using Photoshop, [06:02] You can literally, with Midjourney, you can prompt it, but in Photoshop, you can literally change the pixels. And so figuring out ways to give people finer grained control is a really interesting challenge in generative AI. [06:16] Yeah, absolutely. And I think one really important point here is that there's a lot of different dimensions of how people gain agency. And some of them are things surrounding the AI and are not core to the AI itself. So one example following on with the hotel analogy, right, is you said, maybe everybody has a general contracting team that they can bring to their hotel room. If you show up at a hotel with a construction crew, you're going to be able to do that.

[06:41] they're not going to let you, you know, tear down your room and rebuild it. Right. There's like a sort of social agreement or legal framework or whatever around your contract with that entity where it's not really like your ability to move walls physically. That is the constraint. And so to bring that back to software, what I think about a lot is, um, [07:01] If I had a personal team of 100 developers that were working for me, [07:05] I, what could I do with that power? And, um, you know, let's say I'm using Spotify and I'm like, oh, I hate the way that Spotify shows me these playlists. I wish that it showed me them a different way. Um, I don't, I can't. [07:17] edit the Spotify app. I don't have control over that. And so [07:22] And what I'm thinking about a lot these days is how can we reorient the way software is distributed and constructed so that my... [07:30] so to speak, personal dev team can actually do useful stuff for me? And for example, right, maybe they could make browser extensions for me, or maybe they could, if my software is open source and I control it, maybe they could start adding features for me. How can we change the foundations of software in a way that is best positioned to take advantage of the fact that everybody is going to have this power available to them in the coming years? [07:56] Yeah, no, that makes total sense. And it's just making me think of, I teach this course called how to build an AI chat bot. Um, and one of the really incredible things about this course is we take people from not really being able to code to shipping an app in 30 days. And, um, it's really, it's really cool because I feel like there are all these people out there who are just waking up to

[08:26] build stuff in a way that they couldn't before. And a lot of these people are like, okay, it's a product manager who you work with engineers and maybe you took one coding class in college, but you've never felt confident enough to actually go build and ship a React app or whatever. [08:42] And all they really need is a little push to be like, OK, just use ChatGPT. [08:47] Ask it what to do. Here's some sample code and go. And it really changes things for people where one is on day one, they can have a working chatbot with a couple prompts. And that's like a big, holy shit, I cannot believe that I can actually make something. And then what it does is it motivates them to try to learn the underlying fundamentals in this way that's very connected [09:17] have to learn programming, which is like you spend six months learning about loops and if statements and all this stuff. And you're like, I have no idea how this connects to the things I want to make. [09:27] I just think that there's this really quickly expanding horizon of who gets to make things in the world. And there are a lot of people who are within the horizon of being able to make stuff that don't know it yet, but all they literally have to do is sit down with ChatGPT. And that horizon is going to be significantly larger in five years. And I think that's one of the things that you're working on, and I think it's so exciting. Yeah. [09:55] I love that way of thinking about it. You know, for me, the way that I learned best has always been to make first and then figure out what I'm doing second. And I think people, people often end up in these dumb debates that are like, you know, people should be able to make stuff without learning the fundamentals. And then other people are like, it's really important to learn the fundamentals. And my take is, yes, it's important. If you want to go really far, at some point, you're going to have to dig deep and actually learn what you're doing. And that's going to propel you, but that shouldn't be a barrier to you doing anything. And so in my career, you know,

[10:25] in web development, just like throwing together Rails apps. And I didn't really know how to program. I didn't know Ruby. I was just sort of copy pasting and messing around. And I fought through the pain just 'cause I wanted to make and share stuff. And I think you're exactly right that [10:41] the flip that I see when people start using ChatGPT is all of these frustrating moments that before [10:46] It took incredible dedication to fight through because they were so frustrating. And programming is absolutely full of these things. [10:56] You're missing a semicolon, and it takes you three hours to realize that that's why your thing wasn't working. That's so demoralizing. And people who don't have access to a friend or tutor who can help them with that, there's just a huge drop-off chance, I think, at that point. And when I'm using ChatGPT, even now, I'm a skilled programmer, but I'm often working in areas that I don't know as well. [11:16] with me just feels like the default flips to I have flow, I have momentum, and I have this sort of support that's going to make sure everything goes okay. And I am just less worried about getting stuck. And I think that that feeling is really qualitatively different. And I agree when people realize that it's a really exciting flip for them. [11:37] Yeah, it really is. So I think that that sort of brings us to, you know, the main thing that we wanted to do today, which is, you know, [11:46] And most of these episodes, what we're doing is we're going through historical chats. And then we both kind of interact with ChatGPT and we explore something and find things that we never would have found before. And what you suggested, which I think is brilliant, is like, why don't we just build something together? Why don't we build an app?

[12:16] - Thank you. [12:16] What are the specific ways that you're using ChatGPT to achieve these results in just a very, very detailed way? And we'll try to make it so that if you're a programmer, you can get a lot out of it. And if you're not a programmer, you'll be able to understand what's going on and you'll be able to see, like get a little bit of a taste of like, what would it be like to push myself in this way a little bit and try to build stuff? And I think it'd be really cool. [12:37] Yeah, I'm excited to try it out. And maybe just as a preamble, I can talk a bit about how I think about this task that we're going to do. Right. So what I've been trying to do in my work is sometimes there'll be a brief moment when I'm in the middle of something and I have this thought that's like, oh, I wish I just had. [12:55] a software tool that did X. And again, this is getting out of the hotel mindset. And I've been trying to notice those moments and think, wait, like, could I actually, [13:03] make that? And before, maybe that would have taken a day. So the answer is like, no, not worth it. But now if the answer is like five minutes, maybe, right? And so I've been trying to notice those moments and I've come up with a workflow that's very, very simple and crude, but has worked for me a few times for making these sort of quick apps that help me with stuff. One small example of one that I built before is I was prepping for a trip to Japan. And I actually grew up in Japan and I [13:33] somewhat fluent Japanese, but I have some sort of pockets of missing skill, particularly in more formal communication. And I was on these text message threads with some of my mom's acquaintances who I don't know as well. I mean, I needed to sort of text them in a way that was correct to the context. And I was just struggling. And I was doing all these chat GPT chats to try to help me with the translation. And it was really helpful. But I kept hitting these really annoying chat situations where I would be like, that sounds too formal, make it sound a little more casual.

[14:03] oh, that sounds too casual, make it sound a little more formal. And so what I did in the end is I made this UI tool [14:09] that you can do a translation and then there's a little slider to adjust how formal the translation should be. I love that. And that was just way more, way easier, way more fun. And it sort of encapsulated a lot of the wisdom of the system prompt I had been iterating on in a nice GUI tool that I could then just whip open whenever I wanted. And so that's sort of what I mean when I talk about these little helpful software tools that I wish existed just for me. [14:39] together. [14:40] Okay, cool. Well, let's find a tool to make. I know you have some ideas. I've got some ideas. Why don't you start and we'll just kind of go back and forth until we find something we like. Okay. [14:49] Yeah. So one idea I had was I'm a guest on your podcast right now. And so what's like a utility that I would want to have? [14:58] While I'm [14:59] a guest on your podcast. And, you know, I could imagine like, maybe there's like a time bar that kind of shows me like, how are we doing on time? And so I have an ambient awareness of pacing, sort of like, you can imagine it's like keynote presenter mode, but specialized to being a guest on your show. Maybe there's a checklist of like topics we want to, I want to make sure I hit with you and I can check them off. So I have like this aid. I don't know if there's stuff maybe as a host, do you have ideas for what you might want? [15:26] Let's ideate a little bit because I think you're onto something. This podcast is unique in that ChatGPT is a guest in certain ways or it's a part of it. I'd really love to bring it in even more. For example, one thing that I think would be really useful, I often find myself...

[15:47] like wishing that we had a live transcription [15:52] that could then get easily fed into ChatGPT and have ChatGPT respond to what's being said. [15:59] Um, and, um, I don't know if you feel like equipped to do that or if that would take too long, but something like that would be like really helpful. Cause I'm often sort of like, as we're talking, I'm often like writing stuff into Chachubuti and then being like, this is what they said. Like, what do you think basically? And if, if there was an easier way to do that, um, that would be really helpful. I think like one of the constraints is. [16:24] I would... It would work if it was its own interface. It would be even better if it was inside of ChatGPT somehow. I don't... That's probably going to be, like, really hard to do. But, like, just sort of thinking, like... [16:37] Um, the, maybe the, the, the minimal thing is like real time transcription. That's like really easy to either get into chat GPT or to, um, just have, just have GPT, like give some like sample questions or some thoughts like in real time as things are happening that, um, could help push us in new and interesting directions. [16:59] Okay, so two thoughts. First of all, at a meta level, we should get ChatGPT in on this brainstorming session in a second. Let's do it. Because one really important thing is that... [17:09] The moment for chat to get involved is not like when the idea is fully formed. This is a huge part of my philosophy on this stuff is chat GPT is a muse, not an oracle. So it's not.

[17:19] Like, [17:20] we will craft the perfect app pitch and then bring it to ChatGPT. And then in one shot, it will generate the perfect app. And then we're done. [17:28] It's like a designer or an engineer. The earlier you bring them in, the more they can contribute to the ideas and the more context they build up. And so we should do that. Second brief thought is... [17:41] This is actually getting to one of the hardest parts of this process that I think is least well served by CHI-GPT currently, which is you asked me, could we do this? How hard would it be? [17:54] You know, I'm a programmer, and so I have a lot of context in my head for helping to assess very quickly, like on a scale of one to ten, how feasible is what you're thinking of. And I think one of the biggest challenges that chat isn't as good at yet is helping people who aren't programmers figure out, like, [18:13] How hard is this going to be? Is it even like possible? And so when I build these apps myself, currently there's a lot of that background knowledge I have feeding into the process and helping it along. And so one thing I'm excited to explore in the future is can we make these tools better at, you know, basically telling you, actually, Dan, that's not a good idea. Like, I get it. It would be cool, but it's just going to be too much work. How about let's try this other thing instead? That's what senior engineers do, right? So that's the process going on right [18:43] You're totally right. And it's something that I see in my course as well is like people just go into these like, [18:48] People get grand visions. They have these big ideas, right? And that's what motivates people to build stuff. And I'm constantly sort of like being like, okay, your practice is to figure out what is the core smallest thing you can do and just do that and use that as a stepping stone to like build the more complicated thing later.

[19:18] even if it's not your big vision, is the best way to build stuff. [19:23] And, you know, [19:25] This is classic lean startup sort of iterative agile thinking. [19:29] And the really cool thing is chat can lower those iteration cycles so much. So I was prototyping a game at a hackathon earlier this year and [19:39] I was just trying to have fun at this hackathon. I wasn't trying to like work hard, right? And I wanted to prototype different ways the game could feel. And so I would just say to chat, you know, what if the car moved this way? And then I would go off and talk to people, come back five minutes later, I have a new game. I play it. [19:54] And I'm like, oh, you know, that sucks. Let's do this other way. And so I think it's doubly important when using ChatGPT to realize, like, the superpower you have now is that iterations are fast. It's not that chat always has the right answer. It's that getting a new alternative is much faster. And so your goal as a person driving it is to take advantage of that and [20:16] iterate really quickly and try a bunch of stuff really fast and then respond to it yourself and sort of see where it goes so i think we should try that like one thing we could try is just starting with a very minimally scoped app that has like a clock on it or something and then we'll see that working and sort of like we'll have more ideas you know riffing on that and sort of we'll just go from there or something let's do it i love it okay let's do it so here we are we're in chat gbt

[20:46] I got super excited about this whole real-time transcription thing. And I think you're kind of like, that's probably a little too big to do live in, I don't know, we have like 30 or 40 minutes. I think that's just such a really important thing to note is that this happens online. [21:02] all the time in programming. It's a normal thing to be like, I've got this like amazing thing. And then to be like, no, no, we got it. Like we got to pair it back to like something that's useful that we can do in the time we have. And then we can add more later. [21:16] Yes. And so I think we should start with a super small thing and expand out from there. And that's something that, by the way, you know. [21:25] What I'm always thinking about with my system prompts for chat and tools around this stuff long term is people don't just need help programming. They need help with product management and scoping and coming up with the right ideas to solve the problem and design. And ChatGPT can help with all that stuff, too. And that's really important to remember. [21:43] So, so lead us off. So like, how are we going to get started? What are we trying to build? Or maybe like maybe the first step is trying to figure out what we're building with ChatGPT. Tell us how to get started. [21:53] Okay, so we're just going to use this custom GPT. I actually just threw this together and put in a system prompt that I like. Maybe I'll show that before we get started using it so you get a bit of behind the scenes. So, [22:08] here's what this bot basically [22:12] It does. It's a helpful AI coding assistant. [22:15] Make sure to follow the user's instructions precisely because coding is a domain where sort of details matter. And I give it essentially a very specific opinionated stack for the code it's going to output. React TypeScript, Tailwind. This is a very popular set of technologies. It's one that I have a pre-existing template for in Replit, which is this online coding platform. And I tell it, generate all the code in a single file. So basically, it's going to give us some code.

[22:45] does. So this is really important to constrain what it's doing in a particular technical way to get output that we can work with easily. [22:55] I'll note, by the way, copy-pasting code from chat GPT to other IDEs is not the best workflow in the world. It's obviously not how these tools should work in the future. It's just a duct tape together way that you can do it right now that I like to use for quick experimentation. Now, let's talk a bit about the workflow. [23:12] So, [23:13] I think this might be the most important part of the prompt. [23:17] So you get an initial idea from the user. Part two is absolutely critical. Ask the user for clarification on parts of their idea that are underspecified. [23:30] People, so we'll see this play out in a second. [23:34] We're going to ask it to make an app and our idea is going to be vague. And what we don't want is for chat to just run ahead and make up a bunch of stuff and like, you know, make guess what we mean. We want to be in a conversation where it's going to ask us for help and clarification and we're going to clarify what we mean together. [23:54] Then once ambiguities are resolved, we're going to proceed. We'll make a plan for how the code is going to work first. That helps it generate correct code. And then it's going to write the code. [24:02] So that's the basis. That's really good. So I'm curious, like how well in your experience is this? How good is this at following this workflow once you have it in your in your custom instructions for this GPT?

[24:17] I've done a bunch of iterations on this prompt over time. [24:21] My experience has been that asking clarifying questions, it's really good at that. I'm always amazed at how helpful it's clarifying questions are. The hardest task that it's not so good at is... [24:33] Knowing when to... [24:35] ask you to clarify something or whether to just fill it in itself. I try to resolve that by saying, if there's some minor ambiguities, just make assumptions about what the user means and tell the user what your, what, [24:47] how you're filling them in. But sometimes it'll just barge ahead too quickly. And other times it'll be too conservative and asking for too much input from me. It's like, you know, if you're an employee and you have a manager and you're trying to figure out, you know, what are the things I need to ask them about versus I can just go do myself. It's that dynamic, right? [25:04] Mm-hmm. That makes sense. I'm super curious to see how this goes. All right. Let's see how it goes. So I'm going to kick us off with my goal, which is I'm going to say I'm appearing as a guest on a podcast. I want an app that I can have open while I'm on the show to help me keep track of time and remember what I want to say. All right. [25:33] Okay, so great. We're getting clarifications live. [25:41] I'm starting to read them and think through, um, [25:45] my answers, right? So time tracking.

[25:48] I'll just maybe I'll just keep going and answer these questions. Okay, great. So we got answers. Let's go through one by one. And I'm going to think through my answers to these questions. So first, time tracking. It's saying, do I need a simple stopwatch or countdown timer? So I'm just going to say, I want a progress bar. Or I'll say like digital clock, plus progress bar, going from 9am to 1030am. [26:18] start and end because it's an app literally just for this one appearance on this show. And I'm with you today, this morning from nine to 10 30 AM. So that's going to be hard coded into the app, right? Um, that's what part of the femoral software, we don't need extra, extra stuff, um, preload notes or add and edit notes during the podcast, I guess like, um, preloading is fine. And by the way, I'm using, um, we have a notion link with notes. It'd be cool if it could display [26:48] show the notion link or do you copy paste in markdown from the notion? What do you think? [26:52] I mean, ideally, it shows the notion in an iframe or something like that. Oh, yeah. Okay. So it's funny. You just used... [27:02] you just used the word iframe. So that revealed that you, uh, you know, are thinking on the technical side of this too. Right. And I think, um, [27:11] We might want to linger there for a second. And this is another instance of what I said earlier, where having background knowledge about programming is a huge accelerant to this process right now. And the reality is...

[27:21] this workflow we're doing, [27:23] in my experience, doesn't work that well for people who have zero programming experience yet. I want to build tools that help get us there in the future, but it's interesting. [27:34] Uh... [27:35] We have our notes in Notion. Maybe we could show those. [27:42] embed those somehow or copy paste them in. I'll let chat sort of make a call on how to sort of procedure that. Yeah, I like that. Styling or layout preferences. Um, [27:56] I like giving it a vibe here. One vibe I had in my mind is I kind of like the keynote presenter mode. [28:01] So I could just say like simple, dark keynote presenter mode. [28:07] Vibe. Simple, dark keynote presenter mode vibe. I like because I feel like when you give it vibe, it like kind of knows to be fuzzy about it rather than like very direct. Yeah, for sure. By the way, quick meta note, I'm using the numbers here, which is just a great shortcut for like responding directly to each of his things without too much typing. And by that, you mean like it gave you four questions and you're using numbers to. [28:31] I'm just using the numbers one, two, three, four to refer to each of them. I don't need to say like for the time tracking thing I want. But, you know, I just reduce extra words. Right. That's actually I didn't know that. [28:42] Thank you. [28:43] Number four, any other features? I think that those [28:46] We could have cool ideas in the future, but I'm just going to say not yet. Start simple.

[28:52] Yeah. [28:52] All right, so let's go. [28:56] See what we get. Based on your requirement. Yeah, go for it. It's going to do the plan first. The plan, this is really important. If you have it generate code without a plan, it tends to get stuck in weird, like... [29:10] Uh, corners and that it can't get out of and, um, having a generate a plan first, just like, you know, a real human engineer doing any coding work, having a plan first really starts. Um, so let's see, it's done. One thing that's interesting is, um, for the notes section. [29:27] It's suggesting a text area where notes can be copy pasted rather than doing the embedded notion iframe thing. So that's an interesting decision it made. [29:36] Absolutely is. I don't, you know, we don't know why it made that decision. Presumably it's because it's easier. Yeah. Yeah. [29:42] We can push it towards the embed later. It is nice that-- I like having this early planning section, because it's a quick way to see what it's about to do. So we don't need to try to read the code. We can just-- it's writing a spec document as a product manager that we get to look at. And this all looks pretty good to me. Current time, progress bar, copy paste text area, a read only mode to avoid accidental edits during the podcast. It's kind of a fun idea. We'll see if it incorporates that. [30:09] Now, let's see. The second thing it did is a pseudocode plan. As a programmer, this is really helpful for me because I can actually look at this and see, you know, if there's anything that looks like it's going to need to be fixed in there.

[30:24] So while it's writing the code, we can review that. [30:28] Thank you. [30:29] And it's not just useful for you. I think it's also useful for ChatGPT to write out the product spec, write out the pseudocode, and then actually write the code. And in the same way that it's useful for a programmer to write pseudocode, I think it's useful for ChatGPT so it doesn't go off the rails. It can plan what it's going to be doing and all that kind of stuff. [30:50] Yes, absolutely. [30:53] My general philosophy about this is... [30:56] Just like humans need to plan the right amount for the magnitude of the task, [31:02] As a skilled programmer, I can go do a super quick [31:05] bug fix or tiny feature without writing a whole doc explaining what I'm going to do. But if I'm going to spend six months on a huge project, I better have a plan when I start. And so I think similarly about like, um, [31:16] I try to calibrate my sense of how good is ChatGPT at this task. And the worse it is... [31:22] the more planning is going to be needed upfront and the more coaxing to make sure we sort of gently approach the goal. And so coding for me, this is sort of the right level of planning, I think, for this. You know, this is not a huge app. It's a tiny, tiny little thing. So I think we haven't needed that much iteration. [31:39] Okay, so we got this code. I'm not going to read it. I never read this stuff. I just cut the place and go. Why don't you read it? Tell me why. It's a waste of time. Why would I spend time reading this? Yes, I can read this, but it's dumb. I would rather just see what it did and use the thing. That makes sense. Maybe I'll read it later. Okay, and you're flipping to Replit. So explain to us what this is, what Replit is, and how you got, what are all these files? Give us a background.

[32:09] is an online coding environment where you can, [32:14] set up a quick environment to run any kind of code, and then you can edit it in your browser and have it running. And I can also share this link now with other people who can collaborate with me in this coding environment and use the sort of resulting application. The reason I use it is because it's... [32:34] It's just one quick way to get running without a bunch of fuss and ceremony. And before this, [32:41] interview, I set up just a little template that has sort of the tech stack that I told ChatGPT to use. It's pre-installed here and ready to go. I can share that as like a template with you, you know, as a link or something. But that's all I have going on here. [32:58] anyone who's non-technical who's watching this to realize like this might look complicated, but A, Replit is like it's very templatized. It's very easy to just click a button and get something that looks like this and know where to paste your file and B, like... [33:14] You can actually, like everything that we've done so far is pretty natural language, but [33:19] you can actually get this far even with very little programming knowledge. And, um, and so if it feels intimidating, it's, it's really not, not that hard. And I think that's one of the wonderful things about this. [33:33] So with that, let's paste in our code and see what happens. I'm just pasting from chat GPT.

[33:39] should update live this little window here. Oh my God. Look at that. Look at that. So, um, [33:49] It has the time. [33:51] It's correct. The time is the current time. It's going from nine to 1030. I'm just eyeballing it. That looks right, right? It's 950. The halfway mark is 45 minutes. Okay, awesome. It did the simple dark theme pretty well, I would say. [34:07] Yeah. [34:08] And then let's see, can we type in this box? [34:11] Okay. It has a box for notes. [34:15] Okay. [34:16] Pretty good. Pretty good first draft. This is magic. Like, this is crazy. You know, like we have to like stop for a second and and just just appreciate how wild it is that we live in a time like this where. Yeah. [34:30] Like the amount of time and knowledge and effort that it would have taken just a year ago to do the same thing is sort of like mind blowing. It's like stunning how different this is. Like, like obviously the code that the eventual code that has been written by ChatGPT is like not that hard to write for like a senior programmer or whatever. But like still, you know, it might take an hour or two or whatever, even for even for someone who knows what they're doing. And this did not take an hour. It took five minutes. [35:00] And it also didn't require a senior programmer to do like – [35:04] You didn't write any code. You didn't even look at the code. You literally just typed, "Here's what I want," and asked some clarifying questions. And now you have an app. The thing I love about this is it's styled and it has this cool progress bar that updates live. That's crazy. Yeah, it's so exciting.

[35:19] I want to analyze this from on the one hand, I couldn't agree more with you. It is so cool and exciting, even for me as a programmer. Like you said, I wouldn't have spent the time to make this thing. And the fact that it's ten or a hundred X faster for me to make increases the number of things I'm willing to make. So it's really exciting. And, you know, definitely. So that's awesome. [35:38] I will note, like, again... [35:41] we're going to push it further and see where the limits are now. But like we operated within a simple starting point and we're, [35:50] As we push this thing further, we're going to hit more challenges. So we haven't solved all the issues yet, but I agree, it's a very exciting starting point. One little thing I'll point out... [36:00] That's interesting about this app right now as an example of some of the drawbacks. [36:04] This stack that I tell ChatGPT to use as my simplest sort of microapp stack, it doesn't save your data at all. So actually if I reload this, [36:14] I can open this in a new tab. So here's the app full screen, I guess. And I can have notes here. But if I ever reload, they're lost. So that's... [36:28] I know that limitation as the person who put together this tech stack. And one of the reasons I chose that stack is intentionally because saving data is [36:36] often makes apps way more complicated. Now, I have other stacks I can play with that I haven't set up for this that can save the data. But when I'm thinking of ideas in my head, I'm thinking a lot about like, how can I scope this thing so I don't need to have that technical complexity? And that's a that's a skill that, you know, engineers and PMs, you drill a lot, but normal people haven't thought about as much.

[37:02] You know, I think it points to both, of course, you know, [37:06] Um, [37:07] In another version of this, we should have a GPT that can make apps that can store data. That's a no-brainer. But there's a broader point, which is this negotiation of complexity of implementation and requirements is a delicate dance that we're going to have to work towards better tools that help people do that. I think you're right. I also will note, so one is, so definitely, like saving state and doing logins and authentication, for example, are some of those things where you're like, yeah, that's a little bit more complicated. [37:37] . [37:38] Two, I think that those things are getting easier and easier over time. And I can imagine because this is a single use app, you don't even need a database. You can just save something to a notes file or save it in local storage or whatever. So I think that makes it easier. But there are also services like Supabase or all these other things that it just makes it super, super easy to interact with the database. And what I don't want people to come away with is the idea that I think you're right. [38:08] Or maybe I can see that we know, okay, like saving say, it's going to be like hard. So we're going to scope away from that for now. [38:15] And that definitely saves us time. But even if you're someone who's using ChatGPT who doesn't know that, um... [38:22] You're going to find that out because you're going to be like, hey, can we save state? And then you're going to get stuck for like four hours. And that's actually the only way to learn. So it's actually not bad that you don't know that. It's just that like, I think what you're actually saying is not that like...

[38:35] Normal people who don't have a lot of programming experience can't use ChatGPT for this. They can. It's just it will take them longer and they will run into problems. It's not like ChatGPT is going to solve every problem for you right now. There are still things that experienced developers are going to know and be able to push the chat to do and not do that is going to save them even more time than for a normal person. But it's still quite useful for someone who has very little to no programming experience. It just... [39:05] Yeah, it sometimes is not going to solve every problem for you. Just some of the problems. Yeah, that's a great point. I'm not trying to discourage anyone from using this workflow if you don't know how to program. It's just be curious. Ask for how hard it would be to do various things and try to get those answers. One quick plug I'll give is the research I'm doing. I work at an independent research lab called Ink and Switch, which has pioneered a software architecture called Local First, [39:35] like data privacy and ownership, [39:38] But actually, one of the most interesting benefits for this use case is that we're trying to make it really, really simple to make an app that stores your data, can sync it with other people where you can share data, have it all saved privately. And that shouldn't require setting up some server that you have to run and thinking about operations. It should literally be one line of code and you're off and running, you're storing data, you're syncing it.

[40:08] emerge, where when you make an AI generated app, you should be able to plop it into that environment. Instantly, you get authentication, data sync, data storage, all out of the box for free. No servers in the mix. And I'm really excited about that as one specific future thing to make it easier to build really full functioning tools that go beyond just a little stateless thing. That's really great. [40:38] Yeah. [40:38] If I have my notes in notion, um, [40:43] Maybe we'll try embedding this thing, but the app should be able to maybe load my Notion notes directly into this view in a different way and access those existing notes I have without me needing to copy-paste them. That's the thing I want for the future. But maybe, do you want to keep going with this demo and see if we can get Chachibiki to save our notes in this? Let's do it. Yeah, that would be great. Okay, so... [41:05] Let's give it a little bit of encouragement. This is a great start. Always got to encourage the AI in case it takes over. [41:14] Let's be nice and gain brownie points. [41:20] But actually, also, I find that I want to tell it that it's on the right track so it doesn't freak out and totally redo everything. [41:34] uh entered in the browser so they don't get lost when we reload and i i want to point out like that's a question that anyone can ask i'm sure you have some thoughts about the way that it should do this but um we'll see how good chat gpt is at finding the answer that you would have gone with

[41:53] Exactly. And it said certainly it said an answer. It's going to say we're going to save them in local storage. As an engineer, I know that that's a perfectly fine way to think about it. And then it's just going to output all the new code. So generally, [42:08] For little tools like this, I just tell it, the system prompt, I think, says to just output the entire new code. And that's really like the simplest thing you can do. And it's nice because what I'm going to do is just copy this and paste it into Replit again once it's done. And we'll see what it changed. Sometimes for larger things, I like to tell it a different prompt, which is to generate minimal edits and only tell me the parts that change just because it's faster. [42:38] copy pasting, and that can be a bit annoying. OK, so I pasted in the new code. There's no visual difference. I'm going to reload to make sure we got the new version. These are my notes. OK. [42:51] Let's reload. [42:54] It works. Amazing. There it is. And if I come over here and reload. Oh my God. Um, [43:03] Ah, hello. Reload. Okay. The notes are saved. That's amazing. That's amazing. And one thing that we should be clear on, like that website is live. Like you can send me that URL and it's live. It will not save the notes. I'll have my own version of the notes. So we each have our own notes rather than there being one common note, which is an important detail to point out.

[43:26] It's funny, too. You just, again, used your programming knowledge to say that. We could have also asked chat, does everyone... [43:35] using the app share one notes saved or does everyone have their own? Um, [43:44] Okay, so every user has their separate and unique set of notes. Great. Okay, so I just... There's something really interesting going on here for me, like... [43:54] this is such a different way of thinking how to build programs, not because it's just AI built, but like, [44:01] For example, having everything be in one file. [44:05] or saving things locally. [44:09] is just it's a it's a different paradigm from the current way that we build SaaS apps where it's like microservices or like breaking up into modular components and like having having databases and authentication or whatever. And it's like it's this fundamentally new thing that is only possible because you want to build bespoke apps for like small use cases that are used by single users. [44:33] people. [44:35] And that's actually just like a new world that is starting to blossom a little bit that we don't know where it's going to go. But I just want to underscore for people how much... Okay, so there's this new workflow, which makes people incredibly more productive. It gives people who can't code the ability to build stuff. So it's taking people from zero to like 10, right? And then it also takes people who can build stuff and it makes them...

[45:01] I don't know, 30% faster or whatever. But I think what's really important that you mentioned earlier is it's not just about like [45:07] speed upgrade, it makes it possible for you to build things that you just wouldn't have put the time into before. You just wouldn't have made this before at all. Maybe you make 10 or 20 more things in the world per year because of this that you wouldn't have made previously. Big performance workflow difference. Then also, it's like... [45:28] It's a different kind of software. It's just a different, it's a new thing. It's, there's, there's something coming now that's different from SaaS that is enabled by exactly this stuff. And we're just starting to see what it's like. And this is like, this is like the, the, the earliest inklings of that. And it's so cool to see it. [45:47] Exactly. One thing I love, you know, if you look at this app, it's so simple. It has nothing I don't need because we're building it up from zero. And I've been finding in these, building these little tools for myself. [46:15] But in my experience, a lot of the time, [46:19] It's actually faster now for me to just make the thing that has the tiny features that I need than to go like try to Google which one is the best and learn how to use it. And I love the pared down kind of like nothing is there unless I specifically added it to quality of these micro tools.

[46:36] Yeah, that's really cool. You know what this reminds me of? Have you been looking at what Jason Fried and DHH are doing at 37 Signals? They have this new, I guess, software company underneath 37 Signals. It's called Once. And basically, it's like, [46:53] It's on-premise software for consumers. It's like they're making some sort of app for prosumer business people where when you buy it, instead of paying a SaaS fee, you pay once and then you get all the code. The code is not minified or bundled or whatever. It's just like this. It's mostly in one or a few files and you can read it. And it's owned by you, modifiable by you. And I think you're pushing this forward from a research direction. What's possible to enable this stuff? [47:23] sort of like thinking about, okay, how do you commercialize this kind, this new kind of software? How do people pay for it? And all that kind of stuff. I think that's really interesting. [47:31] Yeah, I love that sort of direction. And in my research at InconSwitch, we're pushing towards, again, this new kind of software distribution platform where you own your business. [47:44] tools and you have the capability to modify them. That's another quick point I want to make that's really important to me is that starting from zero and starting from scratch isn't always the ideal workflow. Often what I actually would prefer is to start with an app that someone else made that's almost right, but I just need to make a small tweak. That's something that's much harder in today's software world because often, like I said earlier, you're not allowed to make that tweak. You can't do it because of infrastructural reasons. And so what I'm showing here is kind of like the easy path that works in the current world, which is just to start over from scratch.

[48:14] But I think actually in the future, there'll be less starting over from scratch and more sort of modding and remixing. And that's going to be possible once. Imagine I send you this repli link. You can actually take that code. One thing I do sometimes in chat is like, [48:30] You could make your own ChatGPT session with this bot and say, hey, I'm starting from this app and paste in the code and say, what I want to do is add this other feature and it will just keep going. [48:42] Pick up that process where I left off and it's in your hands now. And someone else might do it. Yeah. Do you want to try it? Let's do it. [48:50] Okay. So do you want to simulate it where I make a new chat basically, or do you actually want to do it? No, no, no. I want to do it. Okay. Okay. Yeah, yeah, yeah. Let's do it. I love that. So send me the REPL and then I need a link to your custom GPT. So I have the same stuff. [49:08] Yes. So, oh, this is going to be fun. So we got the REPL. [49:13] Thank you. [49:15] Cool. [49:15] So you should be able to see my screen now, right? Yep. Okay, cool. So you took that REPL that you just made, and the REPL is basically just all the code we wrote with ChatGPT, and you sent it to me. And what's really cool is with REPLit, I can either press run, and it'll just run it for me, and that's really great. This is a live thing that's just running on the web right now, or I can just press fork, and I can make... [49:43] Podcast guest Dan's version, way better than the old version. And then I can press fork.

[49:52] And now I have all of the code that you wrote with ChatGPT, and it's somewhere-- Look in the source. It's in source, okay. App.tsx. So it's right here, and that might be a thing that would take you a little while to figure out, but again, you could just throw all of this folder structure into ChatGPT, and it would be like, this is probably where the code is. [50:17] And so now we've got to figure out what we want to do. Like we have, um, we have this sort of initial thing that has a timer and, um, uh, and has, has a note section. Uh, yeah. [50:32] Okay, let's brainstorm a little bit. One thing we could do is we could have a little window that takes the notes and then asks GPT-4 to come up with sample questions. [50:48] Ooh, yeah, that's ambitious. I'm into it. [50:54] I mean, I think that's pretty doable, right? Because like, I mean, really, all we're going to have to do is like, take the notes. Well, maybe we should ask ChatGPT, but like, take the notes, send them up to GPT-4 with a little prompt, and then just display the answer somewhere. [51:18] we finesse the API call to OpenAI correctly is something that I sometimes find is...

[51:25] Yeah. [51:27] Working with APIs can be a bit challenging because... [51:31] APIs change, it might not know about the latest format. Something that's going to come to mind is it's going to be really helpful if we give it like a successful example of an app that actually did a good job calling the OpenAI API. So we could try that pattern of I could send you a link to another REPL where I've already done a good OpenAI call and we could tell it like use this app as sort of like a sample. [52:01] is just paste the GBD4 API docs in there so it knows... [52:04] it knows how to call. Let's try to get that. And it should, I should have web browsing, but let's just try it. So, [52:10] I'm going to say something like, and you tell me if I'm doing something wrong, but I'm going to say something like, I have a web app that I use to track my podcast recordings as I'm doing them. And I want to add a feature. Here's the code I have so far. So I think just giving GBT the context is probably pretty important. [52:37] Yeah, absolutely. So I'm just going to throw it in there. [52:40] and see what it says. [52:43] I [52:44] Yeah, so, okay. I want to add a button. So it's basically saying, great, I see you have a basic web app for tracking podcast recordings. Could you please specify what feature do you like to add to your app? I want to add a button that says generate questions where it takes the podcast notes and then sends them to GPT-4.

[53:14] and displays [53:18] the list of questions that GPT-4 returns. And then the little note here is... [53:26] When you're working with APIs, as you said before, and an API is like... [53:30] It's the GPT-4 service, right? The GPT-[redacted address] that you interact with it, and that changes over time as they make updates. And ChatGPT is not always aware of all those changes. So the best way to work through that is to just go to openai.com or wherever the API is and look at the documentation for the API and just give it to... [53:55] just give it to the... [53:58] to ChatGPT so it knows, okay, here's like actually how we, how we, [54:05] Call the API so [54:10] Let's see, I think this is going to be good. [54:13] So I just found an example of like, [54:15] how [54:16] Um, [54:18] how the gp4 api is struck is structured and i'm just going to paste it one quick note is you just copied some python code dan and we're in javascript so that's a mistake i'm just used to i'm used to working in python um you know this is a thing that like uh again you if there's some programming knowledge to even know which of these things to copy right but i think a node one is gonna go better yeah that's gonna go better okay so thank good good note good note uh i'm just gonna

[54:48] Yes. [54:49] Thank you. [54:54] So basically it's saying, okay, it's reflecting back to me. Here's what I think you want to do. Add a button to the existing UI, create a function that sends the notes to GPT-4, and then display the received questions in your app. And then it's creating a plan. So add a generate questions button, create a function that takes the current notes, sends those notes to the OpenAI API. [55:13] and handles the API response to extract the questions. [55:16] create a new state variable for storing the questions and update this state with the received questions, then show the questions in your UI. That all sounds like pretty reasonable to me. And I'm going to do what you recommended and just copy paste. Yeah. Yeah. Quick warning. It did something interesting, which is that it elided some code. It said existing code in a few spots. So it actually hasn't given us a full new file that we can just copy paste. [55:46] and just telling it, don't emit existing code. Just write a full file that I can copy. Let's see if that does that better. [55:56] Thank you. [55:56] And we're also going to have to set our API key, which it's reminding us to do, and I have an API key I can use. [56:02] And for people that don't know, the API key is just, you have to identify who you are when you make requests to GPT-4 so that OpenAI knows who to charge. [56:13] And so you have a key. [56:14] This is another good example of a tricky thing where like, um,

[56:18] I can see a problem in this code already that it's, um, [56:22] It's going to try to use an API key from assuming that it's running on a server because OpenAI usually their docs assume you're running like a server that's querying them. Whereas in this case, we're running an app in a browser and we have to set up the API key a slightly different way. That's like kind of a... [56:39] That's a subtle thing in some ways that can trip you up if you're not aware of that. Right. And again, like, it's one of those things where we can try this. We'll see what error we get and we'll throw it into ChatGPT and see if it can solve it. If not, we'll just solve it so that we can skip to the fun part. But you're right. There are these little micro moments where it's like, it's not quite right. And it's helpful to like... [57:01] have some sense of what, um, what you're doing, um, so that you can, like you said, you know, it's, if you don't know what you're doing, you can just see what happens and, um, you know, [57:12] Yeah. Tell it to fix it so that it's often fine. It's just that if you know what you're doing, you can avoid some rabbit holes and make faster. All right. So here we go. We're going to copy the code. We're going to copy the code. We're going to hold our breath. We're not going to read the code. And we got an error.

[57:42] Sorry, this is a JavaScript. You don't want that. Probably npm install. OpenAI will do it. I'm just like, I'm sort of perverting to my Python. We probably could have asked chat for that one, to be honest. What did you say? [57:56] We probably could have asked chat how to fix it as well. We could have asked that. Yeah, that's true. I'm getting impatient. I want to get to the magic. [58:05] So here's what we're going to do. [58:08] copy or we're just going to copy this error. [58:11] and paste it in to ChatGPT and see what we get. [58:15] Thank you. [58:16] Typically occurs when environment variables are not accessible in the client side of JavaScript. This is common in projects with Vite, as it doesn't automatically make environment variables available in the front end. This is a good example of where it's telling us some stuff. We could do it this way. I think that it's kind of going to be a pain to set it up this way. One thing I'm thinking about is, so you could just hard code an OpenAI key right into the code here. [58:46] Like now, if you were to share this REPL link, it's possible that anyone who uses it would get to use your OpenAI key, which is a big problem. You just need to rename the REPLit secret you set to be V to OpenAI key. Match that, okay. This is a good example of annoying programmary stuff that shouldn't be necessary to do, but let's try that. [59:08] All right. Running again, holding our breath. This is how these things really go. It doesn't always work the first time. That's a part of the process. I think that it wants you to put API key around the API key in the error message. Again, we probably could have given that to Chad and had it help us with that. We're running out of time here.

[59:38] because it doesn't like querying it from the browser. But that's okay. We're making a little app just for you, so it's going to be fine. [59:46] All right, I'm going to press run. [59:49] and... [59:51] We're going to move over into our web view. And here it is. We've got a button. It's called Generate Questions. That's pretty cool. Let's see if it works. So what I want to do first, we have a bunch of notes for our interview. So I'm going to take this and I'm just going to throw it in here. [1:00:09] Thank you. [1:00:10] We have our timer counting down. We can see it with the progress. I know. Are we going to make it before the end? Okay, so I pasted a bunch of notes and I'm going to press generate questions. [1:00:22] And we'll see what happens. Okay, let's see what happens. So my guess, my intuition is it's not going to work the first try, but we'll see what happens. Have the dev tools open so you can see if it breaks or something. It says connected. There's no errors. This is tricky, right? Because we have to guess like maybe it's working and it's just taking a while. It's totally possible. It could be that it's going to break. We just need to find out where the output's even. We've got questions. [1:00:48] Wait, oh, that's... [1:00:50] Okay, wait. They made a really weird UI for it. So we have questions. Okay. Here are some prompts to discuss with Jeffrey Lit on your podcast episode. After welcoming Jeffrey to the podcast, ask him to give us a brief introduction about himself and what he does. Dive deep into ChatGVT use cases. We love that.

[1:01:09] Can you highlight some transformations or milestone moments in your professional work due to AI? How do you envision the future of AI in day-to-day tasks and professional workspaces? That's a good question. I really like that. One, you know, I think we don't have time to go too much further on the app right now, but one thing that gets me, so I'm, one thing I love about this process is I'm responding to what it did, right? [1:01:39] or something. Um, and maybe you could like check them off as you go. And that's, that's another idea I just had by looking at this. Um, and I think that starts to point towards like, [1:01:48] Thank you. [1:01:49] Some of the advantages of doing it, so you could have just done this in chat GPT itself and not made an app. And I think for what we have so far, it would have been pretty similar because it's just giving you text. But the further we push into like custom UI for kind of maybe you want to reorder the questions and, you know, you could have it build a UI for dragging and dropping the questions into a different order. All that stuff is what comes to mind now. And that's stuff that like because we're in this custom UI, all of that is possible. And those possibilities occur to me in a way that wouldn't if I was just in the chat box. Right. [1:02:19] I think you're totally right. I also think that there's a world like, you know, ChatGPT already has code interpreter where it can write and run code for you. And I really think there's a world where... [1:02:29] Instead of pasting into Replit, especially for these mini one-off apps, you're just using UI that is constructed on the fly in ChatGPT and then modifying it. And then that becomes a custom GPT you can use for a while. And you can already see this in the Google Gemini demo for their newest language model. It does that inside of Bard. And I really think that's where we're going for a lot of this stuff.

[1:02:57] I totally agree. And in fact, I think often people aren't going to even need to ask, make me a UI that does X. Really, the bot should just decide when I ask for X, it should just pick the best UI for that task, which might be text or it might be a custom whole thing, right? [1:03:14] Yeah, totally. Um, [1:03:17] So this was pretty awesome. We came up with an idea. We built a first version, then... [1:03:26] You... [1:03:27] sent it to me and I made some modifications. [1:03:30] which now we can send back to you and you can keep going with it. You have a bunch of ideas for where to take it. Like this is sort of the future of building stuff. And we did it in like an hour. And I think that's so incredible. [1:03:40] I know it's so fun. And we were able to just kind of have fun and talk while we were doing it. Right. We never got like totally stuck. And I think that's the feeling that I want when I'm building stuff. And I think that everybody wants when they're building stuff of effortless flow. And I'm really excited that I think LLM based tools are great. [1:03:58] sort of on a path to enabling that for way more people and way more situations. [1:04:03] Yeah. Yeah. A hundred percent. So as people are, you know, coming out of this episode, they're inspired, they want to go do stuff. They want to learn more about your research. They want to like, think about the future of, of where this is going. Like where do they find you and what should they read? Yeah. Um, I have a personal website, uh, jeffreylip.com where, um, I post, uh, various research. Um, I wrote an essay earlier this year called malleable software in the age of LLMs where I kind of lay out a bit more of sort of the thinking behind this,

[1:04:33] check out more thoughts. I'll be publishing more research in the coming year or two on kind of [1:04:39] How do we take this workflow we just did and actually make it really awesome and integrated better into sort of a whole coherent work environment with less copy pasting and sort of less dealing with infrastructure? So look forward to that, too. Amazing. Well, I'm super excited when that comes out. I would love to have you back. This was incredible. I feel just I'm like walking on air right now. I got there's so much good stuff happening here. And I really appreciate you taking the time to walk us through this. [1:05:09] Thanks so much. This was really fun.