ChatGDP? Using AI to Write for Economics

The following blog is divided into two parts: Part I has been entirely generated by ChatGPT, Part II was written by Chmura economists and discusses the differences between ChatGPT’s economic writing and an economist’s writing.

Part I: AI Writes a Blog

Understanding GDP: Unraveling Misconceptions and Embracing Accuracy

Gross Domestic Product (GDP) is a crucial economic indicator that provides insights into a nation's economic health. However, misconceptions surrounding its definition, particularly in terms of revenue, can hinder a clear understanding of its true value. This article aims to debunk these misconceptions, shed light on the importance of GDP over revenue for measuring the economy, introduce the value-added approach, delve into the factors involved in GDP calculation, and provide notable data from 2014.

One common misconception is equating GDP with total revenue, assuming it represents a nation's overall wealth. However, GDP is a measure of the value of final goods and services produced within a country, not the total revenue generated. Failing to account for intermediate goods can lead to double counting and an inaccurate representation of economic output. To illustrate this, let's explore a scenario involving a fictitious company called "TechPro."

TechPro is a technology firm engaged in manufacturing smartphones. If we were to measure TechPro's economic activity based solely on revenue, it would not provide an accurate representation of the company's contribution to the economy. This is because revenue includes both the sales of the final product and the intermediate goods used in the production process, leading to double counting.
For instance, TechPro generates $50 million in revenue, but it purchases $20 million worth of components from suppliers. By considering revenue alone, we would count the $20 million of components twice – once in TechPro's revenue and again in the revenue of its suppliers. However, the value-added approach offers a solution.

The value-added approach focuses on the value created at each stage of production, eliminating double counting. By considering only the additional value created, this approach provides a more accurate measure of economic output. In the case of TechPro, the value added would be the difference between the total value of smartphones produced and the value of the components used.

GDP calculation includes several factors: consumption, investment, government spending, and net exports. Consumption refers to the value of goods and services purchased by households. Investment encompasses business investments in machinery, equipment, structures, and changes in inventories. Government spending represents the value of goods and services purchased by the government. Net exports account for the difference between exports and imports.

In 2014, the global GDP reached $77.91 trillion, growing by 2.6% compared to the previous year. ^[1]  In the United States, GDP accounted for approximately 91.5% of the total output, with labor income contributing around 55%.^[2]

Understanding GDP is crucial for accurately assessing a nation's economic performance. By dispelling misconceptions surrounding its definition in terms of revenue and embracing the value-added approach, we can obtain a more comprehensive and precise understanding of economic output. Calculating GDP by considering factors such as consumption, investment, government spending, and net exports provides valuable insights for policymakers, businesses, and individuals alike. So, let's delve deeper, educate ourselves, and make informed decisions based on a clearer understanding of GDP. ^{3]

Part II: Comparing AI to Economists and Evaluating Uses of AI in Economic Writing

Many jobs have the potential to be impacted in the wake of generative AI tools, as discussed in a recent Chmura blog. Among the skills most likely to be taken over by AI is writing. This new wave of technology raised a question among Chmura’s team of economists: Can AI write a blog about economics that sounds like it was written by an economist; and if so, how will the future of Chmura’s writing be affected?  More broadly speaking, can economic writing in general being outsourced to AI tools?

To dive deeper into this question, Chmura decided to use ChatGPT to generate a blog about gross domestic product (GDP) and compare it to a Chmura-written blog from 2016. The purpose of the original blog is to clarify some misunderstandings related to GDP and introduce the different calculation methods of GDP.

Using a rough outline of the 2016 GDP blog, we prompted ChatGPT paragraph-by-paragraph to compose a similar blog. As each paragraph was written, we modified the prompts to make them more specific to obtain an output more like the original blog. Finally, we combined those prompts, and ChatGPT wrote the full blog in one draft as presented in Part I, including transitions between paragraphs, an introduction, and a conclusion. 

The complete prompt to produce the blog is:

In terms of writing style, language use, and understandability, this AI model was able to write at a similar level to the existing blog. Definitions and misconceptions surrounding GDP were addressed in a manner that was easily understandable for non-economist readers. ChatGPT took the blog one step further by including a formalized conclusion paragraph to close the article, whereas Chmura’s blog closed with a summary graphic, but not a statement, to wrap up the key points of the article. 

One drawback from AI writing is that transitions between topics and paragraphs were not as natural as the Chmura blog, making the flow of the article less consistent. For example, AI did not provide any transition between the income and expenditure approaches to calculating GDP, leaving readers to wonder which approach is being discussed. In addition, it did not provide a transition to the discussion of 2014 GDP data. As a result, readers may wonder about the purpose of presenting 2014 GDP data.

Another way AI falls short of Chmura’s blog is the use of credible, easy-to-trace sources. ChatGPT was asked to cite any resources used in the article and returned citations for the World Bank and the Bureau of Economic Analysis (BEA). Upon initial inspection, one critical piece of information jumped out: ChatGPT is unable to source anything after September 2021. Knowing this, ChatGPT was asked to report on GDP data from 2014. Even so, the sources accessed had been modified since 2021, making the data reported by ChatGPT unreliable and difficult to fact-check. Many economic indicators are updated annually, quarterly, or even monthly, so until AI is up to date with the most recent economic reports, it remains impractical to use this tool to generate fully sourced articles. That said, it is possible that paid versions of the model include more recent data and regular updates.

When reporting on economic terms and data, several cases of imprecision arose in ChatGPT responses. Chmura’s blogs are precise, differentiating concepts such as GDP, output, and revenue. The language AI used introduces some imprecision and could lead to misunderstandings. For example, in the finalized blog ChatGPT writes, “Failing to account for intermediate goods can lead to double counting and an inaccurate representation of economic output.” AI should not have used the term “output” in this context as it creates more uncertainty in the definition of GDP and lacks precision. However, this mistake is understandable because it is commonly found in internet resources explaining GDP. For example, the International Monetary Fund writes “[GDP] counts all of the output generated within the borders of a country.” ^[4] The accuracy of this statement is questionable as GDP does not account for black market activity. Chmura economists work to remove such uncertainties from blog posts to ensure readers are receiving the most accurate and understandable information, while ChatGPT fails to do the same. 

The sharing of misinformation through AI tools is a widespread concern, bringing up questions of the ethicality of using AI to draft entire reports. For economics articles in particular, it would be easy for AI to randomly generate numbers for indicators, revenue reports, or even GDP. For example, in the written blog, when asked to report on 2014 GDP data for the United States, ChatGPT provided a number for global GDP in 2014, normalized to an unspecified year’s base dollars with an out-of-date source, leading to information that could not be verified or properly interpreted and understood. Perhaps more concerning is that AI cited “GDP accounted for approximately 91.5% of the total output” from the Bureau of Economic Analysis. Chmura economists were unable to confirm this source, find the original report, or find any mention of the number 91.5% in 2014 GDP data for the United States.

One thought is that this statistic references the percent of output in the United States that comes from black market activities. The black market is difficult to measure but estimates for the current size of the black market in the United States range from 7-12% of total output.^[5]   The 8.5% that ChatGPT suggests for 2014 falls in this range, but due to outdated sources and the uncertainty surrounding measuring the size of the underground economy, the statistic cannot be confirmed. Further, ChatGPT never clarifies that this is the case, and without being able to find the original source of the data, this speculation cannot be verified or trusted. This legitimizes the concern of AI generating erroneous data.

Another ethical concern surrounding the use of AI for writing is plagiarism. It is difficult to confirm whether an article was written by AI or not, as AI does not have an extensive memory, making it easy for anyone to claim work done by AI is their own. Additionally, standards surrounding citing AI and the extent to which AI can appropriately be used as a writing tool have not been established.

While there are many dangers to using AI to write economics articles, it is a promising tool for generating a rough draft following some tips for writing with AI. For starters, experimenting with various versions of generative AI may be useful. Open AI’s ChatGPT was used for this example, but other models offer additional resources. Bing Chat has access to the internet and may be able to provide more up to date information about economic indicators. Further, other chatbots may be more or less skilled at writing or generating ideas. Using a variety of AI tools may lead to a more diverse pool of sources and inspiration. In its current form, AI should not be used to generate full articles, but is a powerful tool to provide ideas, eliminate writer’s block, and edit written reports. Using AI could generate more interesting research questions and create opportunities to expand research fields in previously unheard-of directions.

Finally, the original Chmura blog includes a visualization explaining the structure of GDP. ChatGPT was asked to determine what points would be best supported by graphics, and what information to include in such graphics. Suggestions from ChatGPT included:

• An illustration of the common misconception of equating GDP with total revenue that highlights the differences between the two
• A flowchart or diagram showcasing the production process of TechPro, a fictional generated company used to explain GDP calculation in AI’s blog, depicting the flow of intermediate goods and explaining how the value-added approach eliminates double counting
• A pie chart or graph illustrating the breakdown of GDP by components such as consumption, investment, government spending, and net exports
• A line graph showing the trend of GDP growth in the United States over several years to showcase economic performance
• A side-by-side comparison chart noting the key differences between revenue and GDP, emphasizing the limitations of using revenue alone to measure economic activity

Given the numerous examples of graphics to add to the blog, it is possible that including additional graphics would have enhanced the original 2016 blog, and using AI to generate ideas for future blog graphics may prove beneficial for Chmura economists. ChatGPT is able to generate code examples to be entered into software programs that generate graphs, but it does not produce images or charts itself. Other AI tools can generate charts, graphics and images, such as Chartify. While none of these tools combine writing and image generating elements to be able to develop a full blog with graphics in one prompt, these existing tools could speed up the process and prompt new ideas for content.

In summary, Chmura writers, and writers in general, could benefit from using AI in some stages of the writing process. Generating topic ideas, outlines, titles, and suggestions for graphics, introductions, and conclusions are all ways AI can support writers, without introducing misinformation or violating plagiarism regulations. However, due to various concerns outlined above, AI tools cannot be solely relied upon for economic writing. Regardless of advances in this technology, all Chmura blogs remain authored by Chmura economists and data scientists and supported by trusted, accurate sources.

 --------------------------------------------

[1]  World Bank, "World Development Indicators database," data.worldbank.org, accessed September 2021.

[2] Bureau of Economic Analysis, "National Income and Product Accounts: Table 1.1.5. Gross Domestic Product," bea.gov, accessed September 2021.

[3]  Note: ChatGPT can only access sources prior to September 2021. GDP data accessed then has since been updated. ChatGPT data does not report the base year used to adjust 2014 GDP data for inflation and cannot be accurately contextualized.

[4] https://www.imf.org/en/Publications/fandd/issues/Series/Back-to-Basics/gross-domestic-product-GDP

[5] https://www.investopedia.com/articles/markets/032916/how-big-underground-economy-america.asp