LLMs: Writing Aid, Not Writing Replacement
Writing a doc is one of the most effective ways to gain feedback on your ideas and ultimately consensus on whatever is being proposed. Yet, writing a technical design in a one to six-pager is inherently challenging. LLMs offer a quick solution. With a few bullet points of prompts, Chat GPT, Claude, or Nova can write paragraphs of detail for you. This post will describe why this approach ruins the doc, and how LLMs can better be used to improve your writing.
Jeff Bezos is famously quoted as saying that PowerPoint hides sloppy thinking, the real value is in the notes, so just share those instead... leading to a ban of PowerPoint at Amazon, I think this same thought applies to the usage of LLMs too. Don‘t give me the memo generated by a LLM, give me the prompts that created it. Those are what matter.
PowerPoint presentations and bullet points are easy for the author, but harder for the reader. A six-page memo in narrative format is hard for the author, but easy for the reader. If an LLM is working with a context window that doesn‘t include the background, current state of the system, importance of the problems at hand, and best paths forward within the existing architecture, then it won‘t be able to concisely guide a reader through this. If all of this has been gathered to feed to the model, then most of the work for the doc has been done. Good work! This is often the most time consuming, but most important part of writing.
I find that the act of producing a narrative drives me to consume. In Walden, Henry David Thoreau notes the impact a book can have on one‘s thoughts and ideas.
"How many a man has dated a new era in his life from the reading of a book!" — Henry David Thoreau
When I‘m writing to set technical direction, even before receiving feedback, I find myself having my opinion swayed. By trying to form the right paragraphs, looking for structure, seeing what‘s missing, questioning if my conclusion is supported by sufficient evidence, I‘m compelled to search for more details and answer questions that I didn‘t previously know to ask. This very thing that makes writing hard, is the very thing that makes it valuable and must be preserved. LLMs are an alluring way to fill those gaps, but it‘s often with self-affirming filler.
Improving Writing with LLMs
Despite this condemnation, they are an indispensable tool for improving the quality of what‘s written. Once, when I asked if ‘which‘ or ‘whom‘ was considered correct, it told me I used whom correctly, but to remove the unnecessary comma that followed. More importantly it can be used as the first sounding board for feedback. "Are there any missing details that would improve this narrative?", "What alternatives were not considered", "Are there any structural improvements to make?" can all be used as prompts to help fight any bias from the initial draft before getting any feedback from peers.
I work with many brilliant people, some of whom convey these complex ideas in English, their second language. I think using these tools can be instrumental in this scenario. An LLM can quickly correct minor grammar and structuring mistakes without affecting the ideas of the doc. Better yet, the doc can be written in their native language and an LLM can translate it to English before they share it with others.
This can simplify those pre-meetings when asked "mind if you take a look at this before I share with the team", saving 2-3 people a few hours of time. For almost all docs, these steps that demand additional author investment, but improve the doc are worth taking. If it‘s not worth spending time on, and if it‘s assuredly the right decision to make, why are we even meeting about it? Just go do it.
Moving Forward
To further improve my writing, I‘m taking some key steps this year. First, I will draft all my blog posts without the use of an LLM. Once I‘ve created the content and narrative that I‘d be proud to share, I‘ll use an LLM to help me improve it further. By producing more, I hope to improve my cycle of ideating, planning, executing, and delivering which will help me clarify more ideas and bring better vision to my org. Secondly, I bought Style by Joseph M. Williams and Writing Down the Bones by Natalie Goldberg. LLMs give great feedback, but I‘m hoping these books give me better intuition as to why some writing is better than others. Lastly, I plan to continue reading voraciously. Producing content helps clarify and shape thoughts and ideas, but consuming is arguably more important to introduce novelty and give the basis for new opinions.
Conclusion
In the end, it‘s the ideas and discussions that result which are most important. Use the best tools available to foster dialogue. In my experience, the time and effort spent writing is worth it and when LLMs are used as a shortcut rather than an aid, the true value is lost. If you disagree, I‘d love to hear from you and am glad this human-written post could inspire discussion.