As with all new classes of technology, the surrounding discourse takes time to develop and mature. It should come as no surprise, then, that when given a text input for conversing with large language models like ChatGPT, most discussions revolve around how best to take advantage of that text box.

I would argue, however, that the current fascination with “prompts” and other text-based cues—while valuable today in generating novelty from modern AI—misses the real opportunity. It would be like extrapolating a future of computing based on DOS.

Indeed, text and language are incredibly imprecise abstractions of our thoughts, preferences, and desires.

For example, try to imagine explaining to a chatbot what kind of food you want to eat. You might start with a genre—great, so it's only ever going to recommend you pizza, because you said pizza was your favorite food?

How do you explain, in words and sentences, why you like certain things and not others? Do you like gooey things (but not too gooey, obviously)? Do you like sweet things (in certain contexts)? What kinds of spices do you like? And in what combinations?

How should I design a prompt that will recommend me something to eat for dinner?

Should I start typing: Hi my name is Sean and I like food. In general I like trying new things. Except tonight I don't know if I'm in the mood to try something new (by the way, here is a list of everything I've tried so that you can know what I mean when I say I want to try something "new"). I like pretty much everything—except I don't really like olives. Oh and I don't like certain types of pickles but also really, really love other types of pickles. But that doesn't mean I necessarily need to eat something with pickles, it’s just something to keep in mind. I want something healthy but also I might just be saying that aspirationally, and really what I'd like is something that tastes good but also isn't, like, too awful for me...

This is, obviously, silly. The AI knows—effectively—every dish that exists and every aspect of each of those dishes. But I have no way to access this knowledge because I am limited by language. Maybe I can type a good enough paragraph to get something back, but even then, I'm clearly not tapping into the true power of the thing. Given data about my preferences, recommending something for me to eat should be a trivial task for a modern AI.

So what would the real solution look like? You could imagine (and this is simply one example) an interface that played a binary search "this or that" type quiz with me. It would present various prompts—including ones where I might not even understand the relevant features (i.e., not just pictures of food, but also things that tease out current moods and other adjacent preferences and constraints). Instead of me prompting the AI, the AI would prompt me.

I would also show up to this AI having already synced a bundle of general-purpose personal data on which it could draw to inform its responses—things like my geographical location, spending preferences, historical eating data, known allergies, etc.

Once it had all this information, it could simply send me a push notification with some suggestions. Or, it might ask me a few timely questions (e.g., do you want to go out tonight, get delivery, or cook something at home? How hungry are you right now?, etc.) and then align that short-lived context with all the other, long-lived personal data I have provided.

To expect the AI-augmented search and discovery tools of the future to look like Bing is like imagining TikTok asking you to write 500 words about your taste in video before it recommends you something to watch.

It is easy to imagine consumer AI as a disembodied robot with which we can engage (ChatGPT, the movie Her, etc.). However, the real power of consumer AI will not be in AI as a distinct, conversational agent; but rather each of us “merging” with AI and coming to understand it as an extension of ourselves—as a tool for multiplying our agency and effectiveness.

Consider another popular example of AI: the personal assistant.

It would certainly be amazing if each of us had a bespoke, AI-powered personal assistant. The assistant would take care of scheduling meetings, responding to work tasks, mundane chores, etc. You can imagine having a conversation with someone about getting together for dinner, replying, “looping in [my AI assistant] to find a time and place,” and then getting back to your day.

This is as skeuomorphic a vision for the future as the way in which early versions of Apple’s e-reader app placed titles on digitally-rendered wood shelves.

In a fully AI-augmented future, your “assistant” would not be a distinct entity at all, but simply an extension of yourself. Your brain would grow accustomed to automatically—without even thinking about it—ceding control in these sorts of situations. The assistant—the “digital you”—would take over the conversation whenever it made sense. There would be no handoff or distinction. It would speak as you; act as you. From the recipient’s perspective, all the messages would come from you—some would simply be composed and sent by your AI-brain instead of your human brain.

To build for an AI-enabled future requires leapfrogging beyond today’s patterns and thinking. The technology, math, and computer science powering the underlying infrastructure are only going to improve with each passing day. To build for a world of typing prompts in text boxes may be valuable in the short-term, but the real innovation is over the horizon.

‍