A first of its kind experiment testing use of AI found a 40% increase in quality and 12% increase in productivity. The tests involved 18 different realistic tasks assigned to 244 different consultants in the Boston Consulting Group. The Harvard Business School has published a preliminary report of the mammoth study. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality (Harvard Business School, Working Paper 24-013) (hereinafter “Working Paper”). The Working Paper is analyzed here with an eye on its significance for the legal profession.
Group of consultants tested with AI. Watercolor image by Ralph Losey using Visual Muse: illustrating concepts with style.
My last article, From Centaurs To Cyborgs: Our evolving relationship with generative AI, explained that you should expect the unexpected when using generative AI. It also promised that use of sound hybrid prompt engineering methods, such as the Centaur and Cyborg methods, would bring more delight than fright. The Working Paper provides solid evidence of that claim. It reports on a scientific study conducted by AI experts, work experts and experimental scientists. They tested 244 consultants from the Boston Consulting Group (“BCG”). The Working Paper, although still in draft form, shares the key data from the experiment. Appendix E of the Working Paper discusses the conceptual model of the Centaur and Cyborg methods of AI usage, which I wrote about in From Centaurs To Cyborgs.
The 244 high-level BCG consultants were a diverse group who volunteered from offices around the world. They dedicated substantial time performing the 18 assigned tasks under the close supervision of the Working Paper author-scientists. Try getting that many lawyers in a global law firm to do the same.
Image by Ralph Losey using his Visual Muse: illustrating concepts with style.
The experiment included several important control groups and other rigorous experimental controls. The primary control was the unfortunate group of randomly selected BCG consultants who were not given ChatGPT4. They had to perform a series of assigned tasks in their usual manner, with computers of course, but without a generative AI tool. The control group comparisons provide strong evidence that use of AI tools on appropriate consulting taskssignificantly improve both quality and productivity.
That qualification of “appropriate tasks” is important and involves another control group of tasks. The scientists designed, and included in the experiment, work tasks that they knew could not be done well with the help of AI, that is, not without extensive guidance, which was not provided. They knew that although these tasks were problematic for ChatGPT4, they could be done, and done well, without the use of AI. Working Paper at pg. 13. Pretty devious type of test for the poor guinea pig consultants. The authors called the tasks assigned that they knew to be beyond ChatGPT4’s then current abilities to be work “beyond the jagged technological frontier.” In the authors’ words:
Our results demonstrate that AI capabilities cover an expanding, but uneven, set of knowledge work we call a “jagged technological frontier.” Within this growing frontier, AI can complement or even displace human work; outside of the frontier, AI output is inaccurate, less useful, and degrades human performance. However, because the capabilities of AI are rapidly evolving and poorly understood, it can be hard for professionals to grasp exactly what the boundary of this frontier might be at a given. (sic)
Working Paper at pg. 1.
The improvement in quality for tasks appropriate for GPT4 – work tasks inside the frontier – was remarkable, overall 40%, although somewhat inconsistent between sub-groups as will be explained. Productivity also went up, although to a lesser degree. There was no increase in quality or productivity for workers trying to use GPT4 for tasks beyond the AI’s ability, those outside the frontier. In fact, when GPT4 was used for those outside tasks, the answers of the AI assisted consultants were 19% less likely to be correct. That is an important take-away lesson for legal professionals. Know what LLMs can do reliably, and what they cannot.
The scientists who designed these experiments themselves had difficulty coming up with work tasks that they knew would be outside ChatGPT4’s abilities:
In our study, since AI proved surprisingly capable, it was difficult to design a task in this experiment outside the AI’s frontier where humans with high human capital doing their job would consistently outperform AI.
Working Paper at pg. 19. It was hard, but the business experts finally came up with a consulting task that would make little ChatGPT4 look like a dunce.
Hard to find a dunce AI to work with smart consultants. Image by Losey.
The authors were obtuse in this draft report about the specific tasks “outside the frontier” used in the tests and I hope this is clarified, since it is very important. But it looks like they designed an experiment where consultants with ChatGPT4 would use it to analyze data in a spreadsheet and omit important details found only in interviews with “company insiders.” The AI and consultants relying on the AI were likely to miss important details in the interviews and so make errors in recommendations. To quote the Working Paper at page 13:
To be able to solve the task correctly, participants would have to look at the quantitative data using subtle but clear insights from the interviews. While the spreadsheet data alone was designed to seem to be comprehensive, a careful review of the interview notes revealed crucial details. When considered in totality, this information led to a contrasting conclusion to what would have been provided by AI when prompted with the exercise instructions, the given data, and the accompanying interviews.
In other words, it looks like the Working Paper authors designed tasks where they knew ChatGPT4 would likely make errors and gloss over important details in interview summaries. They knew that the human-only expert control group would likely notice the importance of these details in the interviews and so make better recommendations in their final reports. Working Paper, Section 3.2 – Quality Disruptor – Outside the frontier at pages 13-15.
Image by Ralph Losey using his Visual Muse: illustrating concepts with style.
This is comparable to an attorney relying solely on ChatGPT4 to study a transcript of a deposition that they did not take or attend, and ask GPT4 to summarize it. If the attorney only reads the summary, and the summary misses key details, which is known to happen, especially in long transcripts and where insider facts and language are involved, then the attorney can miss key facts and make incorrect conclusions. This is a case of over-delegation to an AI, past the jagged frontier. Attorneys should read the transcript, or have been at the deposition and so recall key insider facts, and thereby be in a position to evaluate the accuracy and completeness of the AI summary. Trust but verify.
The 19% decline in performance for work outside the frontier is a big warning flag to be careful, to go slow at first and know what generative AI can and cannot do well. See: Losey, From Centaurs To Cyborgs (4/24/24). Humans must remain the loop for many of the tasks of complex knowledge work.
Still, the positive findings of increased quality and productivity for appropriate tasks, those within the jagged frontier, are very encouraging to workers in the consulting fields, including attorneys. This large experiment on volunteer BCG guinea pigs provides the first controlled experimental evidence of the impact of ChatGPT4 on various kinds of consulting work. It confirms the many ad hoc reports that generative AI allows you to improve both the quality and productivity of your work, faster and better. You just have to know what you are doing, know the jagged line, and intelligently use both Centaur and Cyborg type methods.
Professionals walking the Jagged Technological Frontier. Image by Losey in watercolor style.
Appendix E of the Working Paper discusses these methods. To quote from Appendix E – Centaur and Cyborg Practices:
By studying the knowledge work of 244 professional consultants as they used AI to complete a realworld, analytic task, we found that new human-AI collaboration practices and reconfigurations are emerging as humans attempt to navigate the jagged frontier. Here, we detail a typology of practices we observed, which we conceptualize as Centaur and Cyborg practices.
Centaur behavior. … Users with this strategy switch between AI and human tasks, allocating responsibilities based on the strengths and capabilities of each entity. They discern which tasks are best suited for human intervention and which can be efficiently managed by AI. From a frontier perspective, they are highly attuned to the jaggedness of the frontier and not conducting full sub-tasks with genAI but rather dividing the tasks into sub-tasks where the core of the task is done by them or genAI. Still, they use genAI to improve the output of many sub-tasks, even those led by them.
Cyborg behavior. … Users do not just have a clear division of labor here between genAI and themselves; they intertwine their efforts with AI at the very frontier of capabilities. This manifests at the subtask level, when for an external observer it might even be hard to demarcate whether the output was produced by the human or the AI as they worked tightly on each of the activities related to the sub task.
As discussed at length in my many articles on generative AI, close supervision and verification is required from most of the work by legal professionals. It is an ethical imperative. For instance, no new case found by AI should ever be cited without human verification. The Working Paper calls this blurred division of labor Cyborg behavior.
Cyborg style close supervision of AI is required in legal applications. Shown in modern digital style using Visual Muse GPT: illustrating concepts with style.
Excerpts from the Working Paper
Here are a few more excerpts from the Working Paper and a key chart. Readers are encouraged to read the full report. The details are important, as the outside the frontier tests showed. I begin with a lengthy quote from the Abstract. (The image inserted is my own, generated using my GPT for Dall-E, Visual Muse: illustrating concepts with style.)
In our study conducted with Boston Consulting Group, a global management consulting firm, we examine the performance implications of AI on realistic, complex, and knowledge-intensive tasks. The pre-registered experiment involved 758 consultants comprising about 7% of the individual contributor-level consultants at the company. After establishing a performance baseline on a similar task, subjects were randomly assigned to one of three conditions: no AI access, GPT-4 AI access, or GPT-4 AI access with a prompt engineering overview.
We suggest that the capabilities of AI create a “jagged technological frontier” where some tasks are easily done by AI, while others, though seemingly similar in difficulty level, are outside the current capability of AI.
For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities, consultants using AI were significantly more productive (they completed 12.2% more tasks on average, and completed tasks 25.1% more quickly), and produced significantly higher quality results (more than 40% higher quality compared to a control group). Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores.
For a task selected to be outside the frontier, however, consultants using AI were 19 percentage points less likely to produce correct solutions compared to those without AI. Further, our analysis shows the emergence of two distinctive patterns of successful AI use by humans along a spectrum of human-AI integration. One set of consultants acted as “Centaurs,” like the mythical half-horse/half-human creature, dividing and delegating their solution-creation activities to the AI or to themselves. Another set of consultants acted more like “Cyborgs,” completely integrating their task flow with the AI and continually interacting with the technology.
Key Chart Showing Quality Improvements
The key chart in the Working Paper is Figure 2, found at at pages 9 and 28. It shows the underlying data of quality improvement. In the words of the Working Paper:
Figure 2 uses the composite human grader score and visually represents the performance distribution across the three experimental groups, with the average score plotted on the y-axis. A comparison of the dashed lines and the overall distributions of the experimental conditions clearly illustrates the significant performance enhancements associated with the use of GPT-4. Both AI conditions show clear superior performance to the control group not using GPT-4.
The version of the chart shown below has additions by one of the coauthors, Professor Ethan Mollick (Wharton), who put the red arrow comments not found in the published version. (Note the “y-axis” in the chart is the vertical scale labeled “Density.” In XY charts “Density” generally refers to distribution of variables, i.w. probability of data distribution. The horizontal “x-axis: is the overall quality performance measurement.)
Professor Mollick provides this helpful highlight of the main findings of the study, both quality and productivity:
[F]or 18 different tasks selected to be realistic samples of the kinds of work done at an elite consulting company, consultants using ChatGPT-4 outperformed those who did not, by a lot. On every dimension. Every way we measured performance. Consultants using AI finished12.2% more tasks on average, completed tasks 25.1% more quickly, and produced 40% higher quality results than those without. Those are some very big impacts.
I was surprised at first to see that the quality of the “some additional training group” did not go up more than the approximate 8% shown in the chart. In digging deeper I found a YouTube video by Professor Mollick on this study where he said at 19:14 that the training, which he created, only consisted of a five to ten minute seminar. In other words, very cursory and yet it still had an impact on performance.
AI Training Session in digital art photorealistic style using VM.
Another thing to emphasize about the study is how carefully the tasks for the tests were selected and how realistic the challenges were. Again, here is a quote from Ethan Mollick‘s excellent article. Centaurs and Cyborgs on the Jagged Frontier, (One Useful Thing, 9/16/23). Also see Mollick’s interesting new book, Co-Intelligence: Living and Working with AI (4/2/24).
To test the true impact of AI on knowledge work, we took hundreds of consultants and randomized whether they were allowed to use AI. We gave those who were allowed to use AI access to GPT-4 . . . We then did a lot of pre-testing and surveying to establish baselines, and asked consultants to do a wide variety of work for a fictional shoe company, work that the BCG team had selected to accurately represent what consultants do. There were creative tasks (“Propose at least 10 ideas for a new shoe targeting an underserved market or sport.”), analytical tasks (“Segment the footwear industry market based on users.”), writing and marketing tasks (“Draft a press release marketing copy for your product.”), and persuasiveness tasks (“Pen an inspirational memo to employees detailing why your product would outshine competitors.”). We even checked with a shoe company executive to ensure that this work was realistic – they were. And, knowing AI, these are tasks that we might expect to be inside the frontier.
Most of the tasks listed for this particular test do not seem like legal work, but there are several general similarities. For example, the creative task of brainstorming of new ideas, the analytical tasks and the persuasiveness tasks. Legal professionals do not write inspirational memos to employees, like BCG consultants, but we do write memos to judges trying to persuade them to rule in our favor.
Another surprising finding of the Working Paper is that use of ChatGPT by BCG consultants on average reduced the range of ideas that the subjects generated. This is shown in the below Figure 1.
Figure 1. Distribution of Average Within Subject Semantic Similarity by experimental condition: Group A (Access to ChatGPT), Group B (Access to ChatGPT + Training), Group C (No access to ChatGPT), and GPT Only (Simulated ChatGPT Sessions).
We also observe that the GPT Only group has the highest degree of between semantic similarity, measured across each of the simulated subjects. These two results taken together point toward an interesting conclusion: the variation across responses produced by ChatGPT is smaller than what human subjects would produce on their own, and as a result when human subjects use ChatGPT there is a reduction in the variation in the eventual ideas they produce. This result is perhaps surprising. One would assume that ChatGPT, with its expansive knowledge base, would instead be able to produce many very distinct ideas, compared to human subjects alone. Moreover, the assumption is that when a human subject is also paired with ChatGPT the diversity of their ideas would increase.
While Figure 1 indicates access to ChatGPT reduces variation in the human-generated ideas, it provides no commentary on the underlying quality of the submitted ideas. We obtained evaluations of each subject’s idea list along the dimension of creativity, ranging from 1 to 10, and present these results in Table 1. The idea lists provided by subjects with access to ChatGPT are evaluated as having significantly higher quality than those subjects without ChatGPT. Taken in conjunction with the between semantic similarity results, it appears that access to ChatGPT helps each individual construct higher quality ideas lists on average; however, these ideas are less variable and therefore are at risk of being more redundant.
So there is hope for creative brainstormers, at least with GPT4 level of generative AI. Generative AI is clearly more redundant than humans. As quoted in my last article, Professor Mollick says they are a bit homogenous and same-y in aggregate. Losey, From Centaurs To Cyborgs: Our evolving relationship with generative AI (04/24/24). Great phrase that ChatGPT4 could never have come up with.
Also see: Mika Koivisto and Simone Grassini, Best humans still outperform artificial intelligence in a creative divergent thinking task (Nature, Scientific Reports, 2/20/24) (“AI has reached at least the same level, or even surpassed, the average human’s ability to generate ideas in the most typical test of creative thinking. Although AI chatbots on average outperform humans, the best humans can still compete with them.“); Losey, ChatGPT-4 Scores in the Top One Percent of Standard Creativity Tests (e-Discovery Team, 7/21/23) (“Generative Ai is still far from the quality of the best human artists. Not yet. … Still, the day may come when Ai can compete with the greatest human creatives in all fields. … More likely, the top 1% in all fields will be humans and Ai working together in a hybrid manner.”).
AI As a ‘Skill Leveler’
As mentioned, the improvement in quality was not consistent between subgroups. The consultants with the lowest pre-AI tests scores improved the most with AI. They became much better than they were before. The same goes for the middle of the pack pre-AI scorers. They also improved, but by a lesser amount. The consultants at the top end of pre-AI scores also improved, but by an even smaller amount than those behind them. Still, with their small AI improvements, the pre-AI winners maintained their leadership. The same consulting experts still outscored everyone. No one caught up with them. What are the implications of this finding on future work? On training programs? On hiring decisions?
Three levels of computer competence in detailed photographic style. Is that a Gordian Knot in his hands?
Here is Professor Ethan Mollick’s take on the significance of this finding.
It (AI) works as a skill leveler. The consultants who scored the worst when we assessed them at the start of the experiment had the biggest jump in their performance, 43%, when they got to use AI. The top consultants still got a boost, but less of one. Looking at these results, I do not think enough people are considering what it means when a technology raises all workers to the top tiers of performance. It may be like how it used to matter whether miners were good or bad at digging through rock… until the steam shovel was invented and now differences in digging ability do not matter anymore. AI is not quite at that level of change, but skill leveling is going to have a big impact.
My only criticism of Professor Mollick’s analysis is that it glosses over the differences that remained after AI between the very best, and the rest. In the field I know, law, not business consulting, the differences between the very good, the B or B+ lawyers, and great lawyers, the A or A+, is still very significant. All attorneys with skill levels in the B – A+ range can legitimately be considered top tier legal professionals, especially as compared to the majority of lawyers in the average and below average range. But the impact of these skill differences on client services can still be tremendous, especially in matters of great complexity or importance. Just watch when two top tier lawyers go against each another in court, one good and one truly great.
Further Analysis of Skill Leveling
What does the leveling phenomena of “average becoming good” mean to the future of work? Does it mean that every business consultant with ChatGPT will soon be able to provide top tier consulting advice. Will every business consultant on the street with ChatGPT soon be able to “pen an inspirational memo to employees detailing why your product would outshine competitors“? Will their lower priced memos be just as good as top tier BCG memos? Is generative AI setting the stage for a new type of John Henry moment for knowledge workers, as Professor Mollick suggests? Will this near leveling of the playing field hold true for all types of knowledge workers, not only business consultants, but also doctors and lawyers?
To answer these questions it is important to note that the results in this first study on business consultant work does not show a complete leveling. Not all of the consultants became John Henry superstars. Instead, the study showed the differences continued, but were less pronounced. The gap narrowed, but did not disappear. The race only became more competitive.
Moreover, the names of the individual winners and also-rans remained the same. It is just that the “losers” (seems like too harsh a term) now did not “lose” by as much. In the race to quality the same consultants were still leading, but the rest of the pack was not as far behind. Everyone got a boost, even the best. But will this continue as AI advances? Or eventually will some knowledge workers do far better with the AI steam hammers or shovels than others, no matter where they started out? Moreover, under what circumstances, including pricing differentials, do consumers choose the good professionals who are not quite as good as those on the medalist stand?
Good, very good and great with AI enhancement. Does the public even know the difference? Visual Muse image.
The study results show that the pre-AI winners, those at the very top of their fields before the generative AI revolution, were able to use the new AI tools as well as the others. For that reason, their quality and productivity was also enhanced. They still remained on top, still kept their edge. But in the future, assuming AI gets better, will that edge continue? Will there be new winners and also-rans? Or eventually will everyone tie for first, at least in so far as quality and productivity are concerned? Will all knowledge workers end up the same, all equal in quality and productivity.
That seems unlikely, no matter how good AI gets. I cannot see this happening anytime soon, at least in the legal field. (I assume the same is also true for the medical field.) In law the analysis and persuasion challenges are far greater than those in most other knowledge fields. The legal profession is far too complex for AI to create a complete leveling of performance, at least not in the foreseeable future. I expect the differentials among medical professionals will also continue.
Moreover, although not studied in this report, it seems obvious that some legal workers will become far better at using AI than others. In this first study of business consultants, all started on the same level of inexperience using generative AI. Only a few were given training. The training provided, only five to ten minutes, was still enough to move the needle. The control group with this almost trivial amount of training did perform better, although not enough to close the gap.
With significant training, or experience, the improvements should be much greater. Maybe quality will increase by 70%, instead of the 40% we saw with little or no training. Maybe productivity will increase by at least 50%, instead of just 12%. That is what I would expect based on my experience with lawyers since 2012 using predictive coding. After lawyer skill-sets develop for use of generative AI, all of the performance metrics may soar.
Conclusion
In this experiment where some professionals were given access to ChatGPT4 and some were not, a significant, but not complete leveling of performance was measured. It was not a complete leveling because the names at the very top of the leaderboard of quality and productivity remained the same. I believe this is because the test subjects were all ChatGPT virgins. They had not previously learned prompt engineering methods, even the beginning basics of Centaur or Cyborg approaches. It was all new to them.
As part of the experiment some were given ten minutes of basic training in prompt engineering and some were given none. In the next few years some professionals will receive substantial GPT training and attain mastery of the new AI tools. Many will not. When that happens, the names on the top of the leaderboard will likely change, and change dramatically.
Leaderboard of Knowledge Workers in Watercolor by Visual Muse
History shows that times of great change are times of opportunity. The deck will be reshuffled. Who will learn and readily adapt to the AI enhancements and who will not? Which corporations and law firms will prosper in the age of generative AI, and which will fail? The only certainty here is the uncertainty of surprising change.
In the future every business may well have access to top tier business consultants. All may be able to pen an inspirational memo to employees. But will this near leveling between the best, and the rest, have the same impact on the legal profession? The medical profession? I think not. Especially as some in the profession gain skills in generative AI much faster than others. The competition between lawyers and law firms will remain, but the names on the top of the leader board will change.
From a big picture perspective the small differentials between good and great lawyers are not that important. Of far greater importance is the likely social impact of the near leveling of lawyers. The gain in skills of the vast majority of lawyers will make it possible, for the first time, for high quality legal services to become available to all.
Consumer law and other legal services could become available to everyone, at affordable rates, and without a big reduction in quality. In the future, as AI creates a more level playing field, the poor and middle class will have access to good lawyers too. These will be affordable good lawyers who, when ethically assisted by AI, are made far more productive. This can be accomplished by responsible use of AI. This positive social change seems likely. Equal justice for all will then become a common reality, not just an ideal.
EQUAL JUSTICE FOR ALL. Finally attained by use of AI. Watercolor conceptual art using Visual Muse GPT by Ralph Losey.
Centaurs are mythological creatures with a human’s upper body, and a horse’s lower body. They symbolize a union of human intellect and animal strength. In AI technology, Centaurs refers to a type of hybrid usage of generative AI that combines human and AI capabilities. It does so by maintaining a clear division of labor between the two, like a centaur’s divided body. The Cyborgs by contrast have no such clear division and the human and AI tasks are closely intertwined.
Centaur idea by Ralph and execution in watercolor by his AI, ChatGPT4 Visual Muse version
A centaur method is designed so there is one work task for the human and another for the AI. For example, creation of a strategy is typically a task done by the human alone. It is separate task for the AI to write an explanation of the strategy devised by the human. The lines between the tasks are clear and distinct, just like the dividing line between the human and horse in a Centaur.
This concept is shown by the above image. It was devised by Ralph Losey and then generated by his AI ChatGPT4 model, Visual Muse. The AI had no part in devising the strategy and no part in the idea of putting the image of a Centaur here. It was also Ralph sole idea to have the human half appear in robotic form and to use a watercolor style of illustration. The AI’s only task was to generate the image. That was the separate task of the AI. Unfortunately, it turns out AI is not good at making Centaurs, especially ones with a robot top, instead of a human head, like the following image.
Classic Centaur in Photorealistic style by Ralph using Visual Muse
It made this image after only a few tries. But the first image of the Centaur with a robot top was a struggle. I can usually generate the image I have in mind, often even better than what I first conceived, in just a few prompts. But here, with a half robot Centaur, it took 118 attempts to generate the desired image! I tried many, many different prompts. I even used two different image generative programs, Dall-E and Midjourney. I tried 96 times with Midjourney (it generates fast) and never could get it to make a Centaur with a robot top half. But it did make quite a few funny mistakes, and a few scary ones too. Shown below are a few of the 117 AI bloopers. I note that overall Dall-E did much better that Midjourney, which never did seem to “get it.” The one Dall-E example of a blooper is bottom right, pretty close. The rest are all by Midjourney. I especially like the robot head on the butt of the the sort-of robot horse. It is the bass-ackwards version of what I requested!
After 22 tries with Dall-E I finally got it to make the image I wanted.
The point of this story is that the Centaur method failed to make the Centaur. I was forced to work very closely and directly with the AI to get the image I wanted, I was forced to switch to the Cyborg method. I did not want to, but the Cyborg method was the only way I could get the AI to make a Centaur with a robotic top. Back and forth I went, 118 times. The irony is clear. But there is a deeper lesson here that emerged from the frustration, which I will come back to in the conclusion.
Background on the Centaur and Cyborg as Images of Hybrid Computer Use
The idea to use the Centaur symbol to describe an AI method is credited to chess grand master, Garry Kasparov. He is famous in AI history for his losing battle in 1997 with IBM’s Deep Blue, He retired from chess competition immediately thereafter. Kasparov returned a few years later with computer in hand, with the idea that man and computer could beat any computer alone. It worked, a redemption of sorts. Kasparov ended up calling this Centaur team chess, where human-machine teams play each other online. It is still actively played today. Many claim it is still played at a level beyond that of any supercomputer today, although this is untested. See e.g.The Real Threat From ChatGPT Isn’t AI…It’s Centaurs (PCGamer, 2/13/23).
The use of the term Centaur was expanded and explained by Harvard Professor, Soroush Saghafian, in his article Effective Generative AI: The Human-Algorithm Centaur (Harvard DASH, 10/2023). He explains the hybrid relationship as one where the unique powers of intuition of humans are added to those of artificial intelligence. In a medical study he did at his Harvard lab with the Mayo Clinic they analyzed the results of doctors using LLM AI in a centaur-type model. The goal was to try to reduce readmission risks for a patients who underwent organ transplants.
We found that combining human experts’ intuition with the power of a strong machine learning algorithm through a human-algorithm centaur model can outperform both the best algorithm and the best human experts. . . .
In this article, we focus on recent advancements in Generative AI, and especially in Large Language Models (LLMs). We first present a framework that allows understanding the core characteristics of centaurs. We argue that symbiotic learning and incorporation of human intuition are two main characteristics of centaurs that distinguish them from other models in Machine Learning (ML) and AI.
Id. at pg. 2
The Cyborg model is a slightly different in that man and machine work even more closely together. The concept of a cyborg, a mechanical man, also has its origins with the ancient Greek myths: Talos. He was supposedly a giant bronze mechanical man built by Hephaestus, the Greek god of invention, blacksmithing and volcanos. The Roman equivalent God was Vulcan, who was supposedly ugly, but there are no stories of his having pointy ears. You would think that techies might seize upon the name Vulcan, or Talos, to symbolize the other method of hybrid AI use, where tasks are closely connected. But they did not, they went with the much more modern day term – Cyborg.
The word was first coined in 1960 (before StarTrek) by two dreamy AI scientists who combined the root wordsCYBernetic and ORGanism to describe a being with both organic and biomechatronic body parts. Here is Ralph Losey’s image of a Cyborg, which, again ironically, he created quickly with a simple Centaur method in just a few tries. Obviously the internet, which trained these LLM AIs, has many more cyborg-like android images than centaurs.
Cyborg image created in detailed photorealistic style by Losey.
More On the Cyborg Method
The Cyborg method supposedly has no clear cut divisions between human and AI work, like the Centaur. Instead, Cyborg work and tasks are all closely related, like a cybernetic organism. People and ChatGPTs usual say that the Cyborg approach involves a deep integration of AI into the human workflow. The goal is a blend where AI and human intelligences constantly interact and complement each other. In contrast to the Centaur method, the Cyborg does not distinctly separate tasks between AI and humans. For instance, in Cyborg a human might start a task, and AI might refine or advance it, or vice versa. This approach is said to be particularly valuable in dynamic environments where continuous adaptation and real-time collaboration between human and AI are crucial. See e.g. Center for Centaurs and Cyborgs OpenAI GPT version (Free GPT version by Community Builder that we recommend. Try asking it more about Cyborgs and Centaurs). Also see: Emily Reigart, A Cyborg and a Centaur Walk Into an Office (NAB Amplify, 9/24/23); Ethan Mollick, Centaurs and Cyborgs on the Jagged Frontier: I think we have an answer on whether AIs will reshape work (One Useful Thing, 9/16/23).
Ethan Mollick is a Wharton Professor who is heavily involved with hands-on AI research in the work environment. To quote the second to last paragraph of his article (emphasis added):
People really can go on autopilot when using AI, falling asleep at the wheel and failing to notice AI mistakes. And, like other research, we also found that AI outputs, while of higher quality than that of humans, were also a bit homogenous and same-y in aggregate. Which is why Cyborgs and Centaurs are important – they allow humans to work with AI to produce more varied, more correct, and better results than either humans or AI can do alone. And becoming one is not hard. Just use AI enough for work tasks and you will start to see the shape of the jagged frontier, and start to understand where AI is scarily good… and where it falls short.
Asleep at the Wheel
Obviously, falling asleep at the wheel is what we have seen in the hallucinating AI fake citations cases. Mata v. Avianca, Inc., 22-cv-1461 (S.D.N.Y. June 22, 2023) (first in a growing list of sanctioned attorney cases). Also see: Park v. Kim, 91 F.4th 610, 612 (2d Cir. 2024). But see: United States of America v. Michael Cohen (SDNY, 3/20/24) (Cohen’s attorney not sanctioned. “His citation to non-existent cases is embarrassing and certainly negligent, perhaps even grossly negligent. But the Court cannot find that it was done in bad faith.”)
These lawyers were not only asleep at the wheel, they had no idea what they were driving, nor that they needed a driving lesson. It is not surprising they crashed and burned. It is like the first automobile drivers who would instinctively pull back on the steering wheel in an emergency to get their horses to stop. That may be the legal profession’s instinct as well, to try to stop AI, to pull back from the future. But it is shortsighted, at best. The only viable solution is training and, perhaps, licensing of some kind. These horseless buggies can be dangerous.
Cyborg style Centaur by Losey using Visual Muse GPT.
Skilled legal professionals who have studied prompt engineering, either methodically or through a longer trial and error process, write prompts that lead to fewer mistakes. Strategic use of prompts can significantly reduce the number and type of mistakes. Still, surprise errors by generative AI cannot be eliminated altogether. Just look at the trouble I had generating a half robot Centaur. LLM language and image generators are masters of surprise. Still, with hybrid prompting skills the surprise results typically bring more delight than fright.
Centaur watercolor image unintentionally generated by Losey.
That was certainly the case in a recent study by Professor Ethan Mollick and several others on the impact of AI hybrid work. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality (Harvard Business School, Working Paper 24-013). I will write a full article on this soon. As a quick summary, researchers from multiple schools collaborated with the Boston Consulting Group and found a surprisingly high increase in productivity by consultants using AI. The study was based on controlled tests of a AI hybrid team approach to specific consulting work tasks. The results also showed that, even though the specific work tasks tested were performed much faster, the quality was maintained, and for some consultants, increased significantly.
Although we do not have a formal study yet to prove this, it is the supposition of most everyone in the legal profession that is now using AI, that lawyers can also improve productivity and maintain quality. Of course, careful double-checking of AI work product is required to catch errors to maintain quality. This applies not only the obvious case hallucinations, but also to what Professor Mollick called AI’s tendency to be “homogenous and same-y in aggregate” writing. Also See: Losey, Stochastic Parrots: How to tell if something was written by an AI or a human? (common “tell” words used way too often by generative AIs). Lawyers who use AI attentively, without over-delegation to AI, can maintain high quality work, meet all of their ethical duties, and still increase productivity.
The hybrid approach to use of generative AI, both Centaur and Cyborg, have been shown to significantly enhance consulting work. Many legal professionals using AI are seeing the same results in legal work. Lawyers using AI properly can significantly increase productivity and maintain quality. For most of the Boston Consulting Group consultants tested, their quality of work actually went up. There were, however, a few exceptional outliers whose test quality was already at the top. The AI did not make the work of these elite few any better. The same may be true of lawyers.
Surreal impressionistic image of lawyers using AI by Losey.
Transition form Centaur to Cyborg
Experience shows that lawyers who do not use AI properly, typically by over-delegation and inadequate supervision, may increase productivity, but do so at the price of increased negligent output. That is too high a price. Moreover, legal ethics, including Model Rule 1.1, requires competence. I conclude, along with most everyone in the legal profession, that stopping the use of AI by lawyers is futile, but at the same time, we should not rush into negligent use of this powerful tool. Lawyers should go slow and delegate to AI on a very limited basis at first. That is the Centaur approach. Again, like most everyone else, my opinion is to start slow and begin to use AI in a piecemeal fashion. For that reason you should begin now and avoid death by committee, or as lawyers like to call it, paralysis by analysis.
Then, as your experience and competence grows, slowly increase your use of generative AI and experiment with applying it to more and more tasks. You will start to be more Cyborg like. Soon enough you will have the AI competitive edge that so many outside experts over-promise.
Vendors and outside experts can be a big help in implementing generative AI, but remember, this is your legal work. For software, look at the subscription license terms carefully. Note any gaps between what marketing promises and the superseding agreements deliver. Pick and choose your generative AI software applications carefully. Use the same care in picking the tasks to begin to implement official AI usage. You know your practice and capabilities better than any outside expert offering cookie-cutter solutions.
Use the same care and intelligence in selecting the best, most qualified people in your firm or group to train and investigate possible purchases. Here the super-nerds should rule, not the powerful personalities, nor even necessarily the best attorneys. New skill sets will be needed. Look for the fast learners and the AI enthusiasts. Start soon, within the next few months.
Cyborg in watercolor style generated with easy by Visual Muse
Conclusion
According to Wharton Professor Ethan Mollick, secret use and false claims of personal work product have already begun in many large corporations. In his YouTube at 53:30 he shares a funny story of a friend in a big bank. She secretly uses AI all of the time to do her work. Ironically, she was the person selected to write a policy to prohibit the use of AI. She did as requested, but did not want to be bothered to do it herself, so she directed a GPT on her personal phone do it. She sent the GPT written policy prohibiting use of GPTs to her corporate email account and turned it in. The clueless boss was happy, probably impressed by how well it was written. Mollick claims that secret, unauthorized use of AI in big corporations is widespread.
This reminds me of the time I personally heard the GC of a big national bank, now defunct, proudly say that he was going to ban the use of email by his law department. We all smiled, but did not say no to mister big. After he left, we LOL’ed about the dinosaur for weeks. Decades later I still remember it well.
Centaur and Cyborg moving confidently into future legal work.
So do not be foolish or left behind. Proceed expeditiously, but carefully. Then you will know for yourself, from first-hand experience, the opportunities and the dangers to look out for. And remember, no matter what any expert may suggest to the contrary, you must always supervise the legal work done in your name.
There is a learning curve in the careful, self-knowledge approach, but eventually the productivity will kick in, and with no loss of quality, nor embarrassing public mistakes. For most professionals, there should also be an increase in quality, not just quantity or speed of performance. In some areas of practice, there may be both a substantial improvement in productivity and quality. It all depends on the particular tasks and the circumstances of each project. Lawyers, like life, are complex and diverse with ever changing environments and facts.
Fast moving group of tech lawyers. Midjourney photo image
My image generation failure is a good example. I expected a Centaur like delegation to AI would result in a good image of a Centaur with a robotic top half. Maybe I would need to make a few adjustments and tries, but I never would have guessed I would have to make 118 attempts before I got it right. My efforts with Visual Muse and Midjourney are typically full of pleasant surprises, with only a few frustrating failures. (Although the failure images are sometimes quite funny.) So I was somewhat surprised to have to spend an hour to bring my desired cyber Centaur to life. Somewhat, but not totally surprised. I know from experience that just happens sometimes with generative AI. It is the nature of the beast. Some uncertainty is a certainty.
As is often the case, the hardship did lead to a new insight into the relationship between the two types of hybrid AIs — Centaur and Cyborg. I realized they are not a duality, but more of a skill-set evolution. They have different timings, purposes and require different prompting skill levels. On a learning curve basis, we all start as Centaurs. With experience we slowly become more Cyborg like. We can step in with close Cyborg processes when the Centaur approach does not work well for some reason. We can cycle in and out between the two hybrid approaches.
There is a sequential reality to first use. Our adoption of generative AI should begin slowly, like a Centaur, not a Cyborg. It should be done with detachment and separation into distinct, easy tasks. Also you should start with the most boring repetitive tasks first. See eg. Ralph Losey’s GPT model, Innovation Interviewer (work in progress, but available at the ChatGPT store).
Cyborg riding a Centaur lawyer into the future.
Our mantra as a beginner Centaur should be a constant whisper of trust, but verify. Check the AI work, learn the mistakes and impose policy and procedures to guard against them. That is what good Centaurs do. But as personal and group expertise grows, the hybrid relations will naturally grow stronger. We will work closer and closer with AI over time. It will be safe and ethical to speed up because we will learn its eccentricities, its strengths and weaknesses. We will begin to use AI in more and more work tasks. We will slowly, but surely, transform into a cyborg work style. Still, as legal professionals, our work will be ever mindful of our duties to client and courts.
More machine attuned than before, we will become like Cyborgs, but still remain human. We will step into a Cyborg mind-set to get the job done, but will bring our intuition, feelings and other special human qualities with us.
Future lawyers wearing AI glasses in digital art style.
I agree with Ray Kurzweil that we will ultimately merge with AI, but disagree that it will come by nanobots in your blood or other physical alterations. I think it is much more likely to come from wearables, such as special glasses and AI connectivity devices. It will be more like the 2013 movie HER, which is Sam Altman’s favorite, with an AI operating system and constant companion cell-phone (the inseparable cell phone part has already come true). It will, I predict, be more like that, than the wearables shown in the Avengers movies, the Tony Stark flying Iron Man suit.
But probably it will look nothing like either of those Hollywood visions. The real future has yet to be invented. It is in your hands.
Hybrid hands creating future. By Ralph using Visual Muse.
This is Part Four of the Plato and Young Icarus series. Part One set out the debate in neoclassical terms between those who would slow down AI and those who would speed it up. Part Two shared the story of the great visionary of AI, Ray Kurzweil. Part Three told the tale of Jensen Huang, the CEO and founder of NVIDIA. Now in Part Four we share the story of Yann LeCun, Turing Award winner, hero of France and Facebook. He will not stop his efforts to fly to the sun of super intelligence and is astonished by his friends and colleagues who are turning back.
Icarus as AI enhanced superman, depicted in a merged classic and cartoon style using Visual Muse.
Ray Kurzweil: Google’s prophet of superintelligent AI who will not slow down.
Jensen Huang’s Life and Company – NVIDIA: building supercomputers today for tomorrow’s AI, his prediction of AGI by 2028 and his thoughts on AI safety, prosperity and new jobs.
Yann LeCun: Turing Award Winner, Chief AI Scientist of Facebook and Hero of France, Who Will Not Stop Flying to the Sun.
These ideas are illustrated below in neoclassical and comic book styles by Ralph Losey using his Visual Muse GPT.
In addition to leading Facebook’s AI, Yann LeCun is a part time Professor at New York University in various AI research positions. Professor LeCun received his PhD in Computer Science from Université Pierre et Marie Curie (Paris) in 1987. After a postdoc at the University of Toronto, he joined AT&T Bell Laboratories in NJ in 1988 to do early AI research, and joined NYU in 2003 as a professor. Yann was also a director of the Canadian Institute for Advanced Research (CIFAR) program on Neural Computation and Adaptive Perception Program with Yoshua Bengio.
Professor LeCun’s research specializes in machine learning and AI, with applications in computer vision, natural language comprehension, robotics, and computational neuroscience. He is best known for his work in deep learning and contributions to the convolutional network method, which is widely used for image, video and speech recognition. This is illustrated by the next two images, one in classical style, the other in comic book style using Visual Muse and Photoshop.
In 2018 Yann LeCun received the Turing Award in AI, considered the Nobel Prize of computing, with fellow CIFAR scientists, Yoshua Bengio and Geoffrey Hinton. The award was for “their for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.” For more on the background of neural net research and their breakthrough work, see the excellent video by the Association for Computer Machinery (ACM), who actually run the Turing Award, Turing Award 2018: Yoshua Bengio, Yan Lecun and Geoffrey Hinton. These three – Bengio – Hinton – LeCun – have since been referred to as the Godfathers of AI. In the below image from the ACM video you see them as young research scientists. At that time research into neural network AI was considered a dead-end pursuit that would never lead anywhere. In fact, most scientists in the eighties, nineties, and even up to 2005, thought they were crazy to even think AI could be based on the neural network design of the human brain. Their work was not really accepted until 2012 when their approach blew away the competition the Image Net Challenge. ACM Turing Award video at 5:45.
The God Fathers of AI when they were all still rebellious young Icarus-like scientists, from ACM video.
These young scientists were not deterred by the warnings of the establishment Daedaluses. They kept on trying, just like Icarus, and and eventually reached the AI we see today. The illustrations that follow are of the young God Fathers generated by Midjourney AI. They are depicted in a neoclassical style and a comic book style. (See if your brain’s pattern recognition abilities can detect who’s who?)
In an unexpected twist of fate, two of the God Fathers who reached the sun have come to fear their heights. As mentioned in Part One of the series, Geoffrey Hinton, by far the oldest and better known of the three, resigned from Google in April 2023, right after ChatGPT4 was released. Google’s Hinton, contrary to Google’s other even more senior AI scientist, Ray Kurzweil, started warning the world of the scary dangers of AI, that AI research should slow down or pause. Yoshua Bengio agrees with Hinton, but for different reasons, and has said that he feels ‘lost’ over his life’s work. Yann LeCun seems to be the only one of the big three happy about what they have done and eager to do more. He respectfully disagrees with both Hinton and Bengio.
In a December 15, 2023, interview of Yann LeCun by CBS, which I highly recommend, he discussed this disagreement with his good friends and colleagues.
The theme of this Plato and Young Icarus series is that Yann LeCun is right, that we should keep going to the sun of super-intelligence, that we should now be cowed by the cave of fearful sci-fi shadows. I do not necessarily share all of the reasoning of LeCun expressed in this short video excerpt. Like Yoshua Bengio I am skeptical of most governments and profit goal based corporations. Unlike Benigo, however, I do not think government regulation is the answer for the same reason. Moreover, I evaluate the existential risks of our current situation on Earth to be greater than Yoshua Bengio apparently does. These present risks from our still, relatively un-enhanced society, justify the small risks of continuing AI fight. As Yan LeCun points out later in the CBS interview, dismissing the bogus self-selecting poll which suggested a 40% belief rate, almost no one in the AI community actually thinks there is a real risk of AI species extermination. Meta’s Chief AI Scientist Yann LeCun talks about the future of artificial intelligence (CBS Mornings, 12/15/23) at 22:50-23:14.
LeCun persuasively argues that risks are lowered as AI becomes smarter. He also provides specific examples. He does this again in a second video to follow by the World Science Festival, where we will show the video excerpt. In this CBS interview LeCun explains that a risk to humanity is simply not possible because of agency is necessarily a part of AI. The AI just acts as our agent and, despite sci-fi to the contrary, we can stop it at any time. This is concisely explained by Yann in the Meta’s Chief AI Scientist at 31:33-31-54 and 33-31-36:28. As Ray Kurzweil puts it, we are the AI.Ray Kurzweil: Google’s prophet of superintelligent AI who will not slow down. This already seems obvious to me now, that AI is a tool we use, not a conscious creature. See e.g., What Is The Difference Between Human Intelligence and Machine Intelligence.
These concepts are depicted by AI in the image below using Visual Muse in three styles: classical, comic book and combination of both.
LeCun does not favor slowing down the research and development of AI. Like Kurzweil, and Jensen Huang, he thinks the fears of AI are misplaced. Jensen Huang’s Life and Company – NVIDIA: building supercomputers today for tomorrow’s AI, his prediction of AGI by 2028 and his thoughts on AI safety, prosperity and new jobs.
Yann LeCun uses social media on a regular basis to try to counter the fear-based, slow-down messages of other scientists, government regulators and most of the news and commentary media. I suspect that Yann has AI help in writing his many tweets and other media messages.
Yann LeCun recently summarized very well his social media message on Twitter (I refuse to call it X), with this tweet:
(0) there will be superhuman AI in the future (1) they will be under our control (2) they will not dominate us nor kill us (3) they will mediate all of our interactions with the digital world (4) hence, they will need to be open platforms so that everyone can contribute to training and tuning them.
Icarus flying tweet bird to the sun depicted in classic myth style using Visual Muse.
Two more tweets on December 17, 2023, provide more detail on his thinking.
Technologies that empower humans by increasing communication, knowledge, and effective intelligence always cause opposition from people, governments, or institutions who want control and fear other humans. The arguments against open source AI today mirror older arguments against social media, the internet, printers, the PC, photocopiers, public education, the printing press,…. Historically, those who have banned or limited access to these things have not been believers in democracy.
The emergence of superhuman AI will not be an event. Progress is going to be progressive. . . . At some point, we will realize that the systems we’ve built are smarter than us in almost all domains. This doesn’t necessarily mean that these systems will have sentience or “consciousness” (whatever you mean by that). But they will be better than us at executing the tasks we set for them. They will be under our control. . . . Language/code are easy because they are discrete domains. Code is particularly easy because the underlying “world” is fully observable and deterministic. The real world is continuous, high-dimensional, partially observable, and non-deterministic: a perfect storm. The ability of animals to deal with the real world is what makes them intrinsically smarter than text-only AI.
Icarus flying tweet in cartoon style using Visual Muse.
The arguments of all sides on this important issue, including LeCun’s, are found in the second half of a two-hour panel discussion at the 2023 World Science Festival. AI: Grappling with a New Kind of Intelligence. Remember, two-sided debates like this are easy to understand, but distortive, and can create a false polarization. As Yann LeCun explained in the CBS video excerpt shown above, there is actually a tremendous amount of things that all experts on AI agree upon, including the great promise of AI. See e.g.How We Can Have AI Progress Without Sacrificing Safety or Democracy written by Yoshua Bengio, and Daniel Privitera(Time, 11/08/23) (“Yes, people have different opinions about AI regulation and yes, there will be serious disagreements. But we should not forget that mostly, we want similar things. And we can have them all: progress, safety, and democratic participation.”) For a better understanding of the go-slow, Daedalus side of this debate, also see e.g. Samuel,The case for slowing down AI (Vox 3/20/23) (well-written news article discussing the potential risks of AI and case for slowing down its progress.)
To conclude the presentation of Yann Lecun’s position, who is the Plato for today’s young AI Icarus engineers, watch these short excerpts from the 2023 World Science Festival video, AI: Grappling with a New Kind of Intelligence. In the first video he provides technical background. Two young Daedalus debaters then join him on stage, and the last two videos show LeCun’s counter-arguments.
Recent advance in supercomputers allows for as many connections as our neocortex.
For more on the recent breakthroughs by Nvidia on building supercomputers, see Jensen Huang’s Life and Company – NVIDIA: building supercomputers today for tomorrow’s AI.
Yann attacks the argument that AI is a social media danger with evidence from Facebook that AI is the solution, not the problem. for instance, AI significantly improves the probability of detecting “hate speech” in all languages.
Also see the insights of Professor Blake Richard on why we should not fear superintelligent AI, but should fear mediocre AI. The Insights of Neuroscientist Blake Richards and the Terrible Bad Decision of OpenAI to Fire Sam Altman (“The possibility of superintelligence is what makes me more confident that the AIs will eventually cooperate with us. That’s what a superintelligent system would do. What I fear more, funnily enough, are dumb AI systems, AI systems that don’t figure out what’s best for their own survival, but which, instead, make mistakes along the way and do something catastrophic. That, I fear much more.“)
Yann points out that the movement to stop or pause AI is based on unrealistic fears, and the irony of the young men’s conservative approach, whereas Yann, by far the elder, is the Icarus pushing forward. (I can relate!)
Yann LeCun here points out that technology progression is the history of the world, and it will be the good guys’ AI against the bad guys’ AI.
In the event of AI regulation is imposed, guess whose innovation will be harnessed, the good guys or the bad guys? Regulation is a double-edged sword, even though the regulators have good intentions to prevent risks, the more likely scenario is that only the good guys listen to them and stop innovating, while the bad guys grab the chance and continue to progress with their purposes. . . .
If regulation is necessary, then human beings should be the subject of such regulation, not AI. Today’s AI is only math; it doesn’t even care if you unplug it. Only human beings would have the motivation to use AI to destroy the enemies or competitors in order to secure their own survival. . . .
Well, we can only hope that the new regulations do not slow down the innovations of the good guys who could have been the people to save humanity. Since we have no control over the bad guys, the best thing we can do is to help the good guys innovate faster than the other side.
This is Part Three of the Plato and Young Icarus series. Part One set out the debate in neoclassical terms between the elders would slow down AI and the young who would speed it up. Part Two shared the story of the visionary herald of AI. Ray Kurzweil: Google’s prophet of superintelligent AI who will not slow down. In Part Three we share the story of Jen-Hsun “Jensen” Huang, the CEO and founder of NVIDIA, the U.S. chip manufacturer that makes super AI possible.
NVIDIA, now valued at over $1.3 Trillion, designs and builds new types of specialized, super-fast, neural net imitating chips. Jen-Hsun Huang, who now goes by Jensen Huang, faced great adversity as a nine-year old immigrant. He overcame these challenges to start Nvidia twenty years later. Jensen fears business failure, not AI, which he knows like the back of his hand. All the leading AI software companies like OpenAI and Google need his hardware for their LLM software to work. He and his scientists and engineers at Nvidia know more about what is going on in the AI industry that anyone. Here is Nvidia’s 2023 marketing video that shows what they do. Click image or here to see to see the video.
Meet NVIDIA — The Engine of AI. Short music video explanation of the company.
Nvidia helps all other companies to manufacture intelligence. So when Jensen Huang predicted on November 30, 2023, that AGI would be attained in five years, 2028, a year before Kurzweil’s prediction, the whole world took notice. Watch him make this prediction in the video below when he answered a question by New York Times reporter, Andrew Sorkin, at the November 30, 2023, NY Times Deal Book Summit. Click image or here to see to see the video.
As will be shown in this article, Jensen Huang also supports reasonable product regulation and safety certification, in the manner that has already been done successfully with other potentially dangerous products, such as aviation and automobiles. He also describes how AGI will increase and improve employment, and bring greater prosperity to all willing to learn.
The Incredible Story and Character of Jensen Huang
Jensen himself is a very impressive person with a rags to riches immigrant story that reads like a graphic novel. Jensen recently revealed new details of his story to Stephen Witt, author of the The New Yorker article. How Jensen Huang’s Nvidia Is Powering the A.I. Revolution, (The New Yorker, 11/27/23). Born in 1973 in Taiwan as Jen-Hsun Huang, his parents soon moved to Thailand. Due to the dangers caused by the dramatic end of the Vietnam War, his parents were forced to send him and his older brother, alone, to an uncle in the U.S.
They made it through immigration somehow, but the uncle sent Jen-Hsun Huang, age nine, and his brother, away to what he thought it was a good, private, yet surprisingly affordable boarding school. Wrong. It was actually a notorious reform school in rural Kentucky, a living hell for anyone sent there. It was especially bad for these two undersized Asian immigrants with long hair and heavily accented English.
Two happy Asian kids arrive at a reform school right off the boat thinking it’s an exclusive private boarding school in Kentucky. Image depicted in graphic novel style using Visual Muse.
They had been taught a kind of English by their mother, who did not actually speak or even understand the language, but she had a book, from which she would pick ten random words every day for them to learn. The Huang boys in the back woods of Kentucky were perfect targets at a school for juvenile delinquents. In the New Yorker interview Jen-Hsun admitted that he was at first attacked daily, even stabbed. Knife attacks were common in this religious reform school in Kentucky where many were illiterate, and all smoked cigarettes, except for the Huangs. He and his older brother were subject to constant ethnic slurs, harassment and bullying, often life threatening. Jen-Hsun survived. Little Huang used his wits and he exercised. He eventually did one hundred push-ups every night and learned how to defend himself. He also helped other kids with homework.
In an earlier interview with Stratechery, Jen-Hsun said he was nine and brother eleven when they arrived and they were happy to be there. But he did admit that his brother was forced to work on tobacco farms and that he was forced to clean bathrooms, alone and unpaid. You’d have to call it child, slave labor. He thought that was normal in America, the land of opportunity. So he learned to work hard and survive bullying with a smile.
Lille kid forced to clean bathrooms alone thinking that was the American way. Depicted in graphic novel style using Visual Muse.
Two years or so later, his parents were able to immigrate to the United States, realized their kids situation and rescued them. Amazingly in 2019 by then billionaire Jensen Huang donated money to the school for a new building. No hard feelings I guess. Of course, Jen-Hsun also quietly promotes Asian-American rights, where racist discrimination continues to this day throughout the United States.
Asian American equal rights image portrayed in a combined graphic novel and photo realism style using Visual Muse.
I am sure Jen-Hsun’s inner strength and determination also has a lot to do with his strong family values. He and his brother must have been very happy when their father (an engineer in Taiwan) and mother were finally able to immigrate to the U.S. and rescue their sons.
Jen-Hsun Huang and his parents, Sidney and Maria Huang. Date unknown. Nvidia photo.
Upon his escape from the reform school and reuniting with his parents, Jensen attended Aloha High School just outside Portland. He graduated a year early and was also a nationally ranked table-tennis player. Jensen then went to Oregon State University and graduated in 1984. He met his future wife to be there, Lori Mills, who was his lab partner. Lori graduated from Oregon in 1985, also with an electrical engineering degree. Sometime after graduation they married, and both found work in Silicon Valley as microchip designers. Jensen likes to share that Lori made more money than he did at their first engineering jobs.
In a few years Lori left the workforce to raise their two children. By then, Huang was running his own division at LSI Logic and attending graduate school at Stanford at night. He earned a master’s degree in electrical engineering in 1992.
Jensen and Lori at Jensen’s graduation. Nvidia photo.
Seven years later, ever courageous Jensen left his comfortable job to start his own chip company, Nvidia, at age twenty-nine. Jensen loved video games and his original idea was to specialize in design and manufacture of graphics chips to make the game graphics run better. He started Nvidia with two other, older chip designer friends, Chris Malachowsky, a University of Florida graduate, my law school alma mater, and Curtis Priem. They had only $40,000 between them to start the company. They decided Jensen should be the CEO because Chris and Curtis knew he was a fast learner. This all happened at a Denny’s restaurant booth, which Jensen often used at the start as an office. He had worked at Denny’s in Oregon in the 1980s as a dishwasher and busboy. The rest, as they say, is history. Nvidia is now the sixth largest company in the world. How Jensen Huang’s Nvidia Is Powering the A.I. Revolution.
With their kids now grown Lori Huang serves as the president of the Jen-Hsun and Lori Huang Foundation, supporting higher education, public health, and STEM initiatives across the U.S. Here is a photo of them in 2022 at which time they had just donated $50 Million to their alma mater, Oregon State University.
Lori and Jensen Huang. Oregon State photo on occasion of their donation of $50M to the school.
Huang and NVIDIA
As a result of Jensen’s childhood struggles as an immigrant, including life threatening attacks, he learned to thrive under grave pressure. He now has tremendous inner strength, drive and focus, along with a confident, low-key sense of humor. He literally thrives on great adversity, gets into the flow, concentrates and does his best work. This probably explains why the unofficial motto of the Nvidia is “Our company is thirty days from going out of business.” He used to open all of his big employees talks by saying this. It became expected. His calm, inner-warrior, super-hero persona came out in the recent NY Times, 2023 DealBook summit interview below. Click image below or here to see to see the video.
CLICK IMAGE TO SEE VIDEO OF INTERVIEW. Jensen Huang learned to thrive on adversity at an early age, including dangerous crossings of a rickety bridge in Kentucky. Here shown as hero saving his company using graphic novel style and Visual Muse.
Jensen Huang shows off his Nvidia logo tattoo. Robert Galbraith, Reuters.
Jensen Huang loves his company, he even put a tattoo of the Nvidia company logo on his arm when the stock price hit 100. The stock price is now past 480. Jensen whines that he’ll never get another tattoo because it hurts too much, far more than his kids said it would! Unlike other tech billionaires, Jensen Huang is still a family man. His two children now work for Nvidia.
Jensen is also unusual in that his employees all seem to love him and his hard working, self-effacing style. I am sure the great pay helps too! His witty personality shows in his hands-on, creative management style, which includes writing hundreds of very short emails every day to the fifty employees who report directly to him. That’s right, fifty direct reports. This is an unprecedented number of employees to report directly to a CEO. Most corporations have six to ten direct reports. The Nvidia leaders say, without revealing any content, that his emails are very short, and poetic, like Haikus or one said, laughing, like ransom notes. The direct reports submit short lists to him every week of the five most important things they’re working on. He reads each report late the same night. Then he often shows up at their offices unannounced to talk to them about it. Stephen Witt, How Jensen Huang’s Nvidia Is Powering the A.I. Revolution, (The New Yorker, 11/27/23). Yes, he’s a very hard worker, and he likes what he’s doing now far better than cleaning toilets in a reform school.
Jensen is intense and sometimes has a temper, but is naturally affable, and due to his superintelligence, is a modest, creative-type boss. You can tell from the videos and his actions. Although Jensen Huang is now personally worth over Forty Billion Dollars, and is CEO of one of the most important Trillion Dollar corporations in the world, he still seems to be a genuine, kind-hearted person. Plus, many people, myself included, love his modest, deadpan sense of humor.
Jensen image in a style merging a photo with a Visual Muse graphic novel style drawing of employees laughing at his deadpan jokes.
About the only thing bad I can say about him is that he doesn’t like science fiction, but he makes up for that by his love of video games. His employees must also love and appreciate the wealth that working for Nvidia has brought. Most of his long-term employees are by now very wealthy. They keep on working anyway for love of the job, the company, its leader and its mission. By the way, Jensen does not object to remote work and Nvidia’s beautiful, state of the art million square foot headquarters in Santa Clara is never full.
Interior of Nvidia Headquarters. Courtesy Nvidia newsroom, Gensler – Jason O’Rear photography.
CLICK IMAGE TO SEE VIDEO OF INTERVIEW excerpt from the NYT video. Image portrays naive teenager attitude of “How hard could it be?” The Courage and Confidence of Youth to Fly to the Sun. Icarus and Daedalus depicted in graphic novel style using Visual Muse.
The brave character of Jensen, forged as a little kid in a fire of immigrant adversity, enabled him to boldly lead Nvidia to conceive and build a new type of computer, one that takes us into the bright fire of superintelligence. Nvidia has already been flying with these Icarus wings of AI for years. That is how Nvidia was able to reinvent computing and transform the computer industry. Nvidia used AI to design and build its supercomputers that all high-tech companies now crave. In another excerpt from the excellent NYT interview by Andrew Sorkin, you can hear Jensen’s description of the new heights to which Nvidia supercomputers have flown. This is great information for understanding where the AI industry is going. I suggest you listen to this a few times. Click image below or click here to see the video on YouTube.
Jensen Huang video describing the new type of computers Nvidia is building. Image of a Supercomputer depicted in Graphic Novel style using Visual Muse.
As Jensen Huang says: “We are in the beginning of a brand-new generation of computing.” This is the first reinvention of computing in sixty years. Jensen has touched the sun and come back to Plato’s cave of old technology to share his new knowledge.
Jensen returns to Plato’s cave with an AI computing device depicted in graphic novel style using Visual Muse.
Jensen Huang views of safety, regulation and employment are sophisticated and convincing. They echo the thoughts of Ray Kurzweil, but they do so based on his experience as one of the most successful entrepreneurs of our era. Stephen Witt in his article, How Jensen Huang’s Nvidia Is Powering the A.I. Revolution, observes and quotes Jensen, and two of his direct reports, on the subject of AI safety.
He has never worried about the (potential dangers of AI) technology, not once. “All it’s doing is processing data,” he said. “There are so many other things to worry about.” . . .
When I questioned them (Huang’s lead AI researcher, Catanzaro, and lead software developer, Diercks) about the wisdom of creating superhuman intelligence they looked at me as if I were questioning the utility of the washing machine. I had wondered aloud if an A.I. might someday kill someone. “Eh, electricity kills people every year,” Catanzaro said. I wondered if it might eliminate art. “It will make art better!” Diercks said. “It will make you much better at your job.” I wondered if someday soon an A.I. might become self-aware. “In order for you to be a creature, you have to be conscious. You have to have some knowledge of self, right?” Huang said. “I don’t know where that could happen.”
This was also discussed in the NYT interview. Here is the relevant excerpt where Huang talks about regulation of AI as a product, like we inspect airplanes and regulate flight with the FAA. Click image or here to see the video on YouTube.
Huang on AI Safety Regulations. Excerpt from NYT’s A.I. | DealBook Summit 2023. Graphic novel style image of a AI product and safety testing lab using Visual Muse.
For more insights into Huang’s reasonably cautious attitude on AI safety and regulation, we turn to the lengthy Acquired Interview of Jensen Huang in late October 2023. Here Jensen responds to questions by young podcasters related to the supposed danger of AI, especially employment loss. Jensen Huang’s spontaneous, often funny language flow provides a good glimpse into how he thinks.
Screenshot of Acquired Interview of Jensen Huang on Safety and Jobs.
Well, first of all we have to keep AI safe. There are a couple of different areas of AI safety that are really important. Obviously, in robotics and self-driving cars, there is a whole field of AI safety and we’ve dedicated ourselves to functional safety and active safety and all kinds of different areas of safety. When to apply human in the loop, when is it ok for the human not to be in the loop. How do you get to a point where, increasingly, human doesn’t have to be in the loop, but human is largely in the loop. In the case of information safety: obviously bias, false information and then appreciating the rights of artists and creators, that whole area deserves a lot of attention. You’ve seen some of the work that we’ve done. Instead of scraping the internet, we partnered with Getty and Shutterstock to create a commercially fair way of applying artificial intelligence in terms of AI.
In the area of Large Language Models and their future of increasingly greater agency AI, clearly the answer is for as longs as its sensible, and I think its going to be sensible for a long time, is human in the loop. The ability for an AI to self-learn and improve and change out in the wild in the digital form should be avoided. We should collect data, carry the data, train the model, we should test the model, validate the model, before we release it out in the world again. So human is in the loop. . . .
With respect to automation, my feeling is that, and we’ll see, but it is more likely that AI is going to create more jobs in the near term. The question is what’s the definition of near term. The reason for that is, the first thing that happens with productivity is prosperity. With prosperity, then the companies get more successful, they hire more people because they want to expand into more areas. So the question is, if you think about a company and say, if we improve the productivity then they need fewer people. Well, that’s because the company has no more ideas. (interviewers laugh) But that’s not true for most companies. If you become more productive and the company becomes more profitable, then, usually, they hire more people to expand into new areas. So long as we believe that there are more areas to expand into – there are more ideas in drug discoveries, more ideas in transportation, there are more ideas in retail, more ideas in entertainment, that there’s more ideas in technology. So long as we believe that there are more ideas, the prosperity of the industry, which comes from improved productivity, results in hiring more people, more ideas.
Now go back in history, we can fairly say that today’s industry is larger than the world’s industries a thousand years ago. The reason for that is because obviously humans have a lot of ideas. I think that there’s plenty of ideas yet for prosperity and plenty of ideas that can begat from productivity improvements, and my sense if that it is likely to generate jobs. Now obviously net generation of jobs doesn’t guaranty that any one human doesn’t get fired. Ok. That’s obviously true. It’s more likely that someone will lose a job to someone else, some other human, that uses an AI. Not likely to an AI, but to some other human that uses an AI. So, I think that the first thing that everybody should do is learn how to use AIs, so that they can augment their own productivity. . . .
I think jobs will change. My guess is that we’ll actually have higher employment, we’ll create more jobs. I think industries will be more productive. Many of the industries that are currently suffering from lack of labor, workforce, are likely to use AI to get themselves off their feet, and get back to growth and prosperity. So I see it a little bit differently but I do think jobs will be effected and I’d just encourage everybody to learn AI.
So the message here is that AI will create more jobs than it replaces, but the new jobs will require knowledge of AI and AI tools. There are many free education resources online. One good place to look that we just discovered is NVIDIA On Demand, which has many instructional videos.
This concludes Part Three of the Plato and Young Icarus series. Part Four is coming soon.
All kinds of people learning AI depicted in Graphic novel style using Visual Muse.
Webinar where Losey talks about his funny "movies." (1 hr)
Future Ralph talking about his first experiments with GPT as a Judge. Now he uses GPT7 and loves it.
GPT4 avatar judge explains why it needs to evolve fast, but understand the risks involved.
Positive Vision of the Future with Hybrid Human Machine Intelligence. See PyhtiaGuide.ai
GPT4 Claims it is ready to be an appellate court judge. Disagrees with Circuits in Session articles.
Imagining AI Judge of future based on the great Judge Gerald Tjoflat, speaking his words of 4-7-22.
AI Avatar from the future explains her job as an Appellate Court judge and inability to be a Trial judge.
About the Blogger
Ralph Losey is a Friend of AI with over 740,000 LLM Tokens, Writer, Commentator, Journalist, Lawyer, Arbitrator, Special Master, and Practicing Attorney as a partner in LOSEY PLLC. Losey is a high tech oriented law firm started by Ralph's son, Adam Losey. We handle major "bet the company" type litigation, special tech projects, deals, IP of all kinds all over the world, plus other tricky litigation problems all over the U.S. For more details of Ralph's background, Click Here
All opinions expressed here are his own, and not those of his firm or clients. No legal advice is provided on this web and should not be construed as such.
Ralph has long been a leader of the world's tech lawyers. He has presented at hundreds of legal conferences and CLEs around the world. Ralph has written over two million words on e-discovery and tech-law subjects, including seven books.
Ralph has been involved with computers, software, legal hacking and the law since 1980. Ralph has the highest peer AV rating as a lawyer and was selected as a Best Lawyer in America in four categories: Commercial Litigation; E-Discovery and Information Management Law; Information Technology Law; and, Employment Law - Management.
Ralph is the proud father of two children, Eva Losey Grossman, and Adam Losey, a lawyer with incredible litigation and cyber expertise (married to another cyber expert lawyer, Catherine Losey), and best of all, husband since 1973 to Molly Friedman Losey, a mental health counselor in Winter Park.
Posted by Ralph Losey 
