Getting Started with AI: Good Enough Prompting | One Useful Thing
As a New York Times article on the paper reported: “they were treating [the AI] like a search engine for directed questions: ‘Is cirrhosis a risk factor for cancer? What are possible diagnoses for eye pain?’ It was only a fraction of the doctors who realized they could literally copy-paste in the entire case history into the chatbot and just ask it to give a comprehensive answer to the entire question.” […] There are many stumbling blocks: people treat AI like Google, asking it factual questions. But AI is not Google, and do not provide consistent, or even reliable, answers.
- AI’s potential is wasted without the right approach.
- Doctors using GPT-4 for diagnoses performed no better than those without it—and both did worse than GPT-4 alone. Why? They treated the AI like a search engine, asking narrow, factual questions instead of giving it full context and leveraging its ability to synthesize complex information.
AIs are inconsistent and weird, and often have different results across different models. For example, they are sensitive to small changes in spacing or formatting; they get more accurate when you tell them to “read the question again;” they seem to respond better to politeness (but don’t overdo it); and they may get lazier in December, perhaps because they have picked up on the concept of winter break. Add to all of this the fact that getting good results out of AI without a lot of formal training is likely growing easier as larger models are less sensitive to changes in prompting technique than older AIs, and new techniques allow the AI to improve your prompt for you.
You can learn details of prompt engineering and the basics of how LLMs work, of course, but for most people, that is not a required starting place. You just need to use AI enough to get a feel for what you can use it for in your area of expertise. The most important thing to do is to get 10 or so hours of use with an advanced AI system. And to do that you just need to be a good-enough prompter to overcome the barriers that hold many AI users back.
The single most useful thing you can do to understand AI is to use AI. There are lots of reasons people may decide to give up on using an AI quickly, from early hallucinations (the AI isn’t good enough) to existential discomfort (the AI is too good), but many of those initial reactions are tempered over time. Your goal is simple: spend 10 hours using AI on tasks that actually matter to you. After that, you’ll have a natural sense of how AI fits into your work and life. You’ll develop an intuition for effective prompting, and you’ll better understand AI’s potential.
- Prompt engineering isn’t the place to start.
- Instead, the key is spending 10 hours using AI on tasks you care about. This hands-on experience helps you naturally develop the intuition needed to prompt effectively.
I mean literally treat AI just like an infinitely patient new coworker who forgets everything you tell them each new conversation. Two parts of this are analogous to working with humans (being new on the job and being a coworker) and two of them are very alien (forgetting everything and being infinitely patient). We should start with where AIs are closest to humans, because that is the key to good-enough prompting. As it is a coworker, you want to work with it, not just give it orders, and you also want to learn out what it is good or bad at.
Start by using it in areas of your expertise, where you are able to quickly figure out the shape of the jagged frontier of its ability. Because you are expert, you will be able to quickly assess where the AI is wrong or right. You do need to be prepared for it to give you plausible but wrong answers, but don’t let the risk of these hallucinations scare you off initially. Though hallucinations may be inevitable, you will learn where they are a big deal, and where they are not, over time. You can reduce the rate of hallucinations somewhat by giving the AI the ability to be wrong, for example, writing: if you’re unsure or necessary information, say “I don’t have enough information to answer this” can make a big difference.
- Instead of treating AI as an intern, treat it like an infinitely patient coworker who:
-
Is highly capable but needs specific guidance.
Since the AI is new on the job, so you need to be very clear [and detailed] on exactly what you want [in the prompts]. You don’t want a report on the pros and cons in remote learning, you want a report on the pros and cons in remote learning appropriate for a regional university in the Midwestern US and that might convince a business school Dean to fund a new remote learning program.
Other ways to help give the AI clarity is by giving the AI examples of good or bad responses (called “Fewshot Prompting”) and giving it step-by-step directions of what you want to accomplish.
You can also give it feedback just as you would another human being asking for improvement or just request that it ask you questions about anything that is unclear.
-
Forgets everything at the end of each conversation.
Now let’s talk about the less human aspects of AI, like its forgetfulness - the fact that each new conversation wipes the AI’s understanding of your particular situation. As a result, you also need to provide context.
Try roles out, but you don’t need to use them if they are not useful. You can also just give it whatever information you have lying around. Entire documents, instruction manuals, or even previous conversations are often helpful, just remember to pay attention to the AI’s memory, its context window.
-
Never gets annoyed and tired of iterating.
This introduces something new in intellectual life, abundance. You don’t need to ask for one email, ask for three in different tones to inspire you. You don’t need to ask for one way to complete a sentence, ask for 15 options and see if that unlocks your writing. Don’t ask for 5 ideas, ask for 30.
Your job becomes one of pushing for variation (“give me ideas that are 80% weirder”), recombination (“combine ideas 12 and 16”) and expansion (“more ideas like number 12”), before selecting one you like.
-
Working with AI is a dialogue, not an order.
For getting a thinking partner, the key to using the AI is to have a natural dialogue. Just talk to it. Most people find this easiest to do via voice on their phone.
Remember to be kind to AI. Saying “please” can actually improve accuracy by signaling to the model that you’re issuing a command rather than asking a question, which helps narrow the search space.
Develop an experimental mindset when using AI—treating it as a partner to learn from through trial and error.
ChatGPT Prompt Engineering for Developers - DeepLearning.AI by Andrew Ng
Basic Invitation Structure: “Act as a [ROLE] perform [TASK] in [FORMAT]”
Tasks
- Pushing you to think through your own ideas
- Use AI as a rubber duck
- Critic: “Critique my argument: [argument]”
- Writing a cover letter
- “Write me a personalized cover letter explaining why I’m a great candidate for this job. The job title is [Job Title], the company is [Company Name], and here is the job description: [Paste Job Description]”
- “Revise and personalize this cover letter using my resume: [Paste Resume Content]”