Data Science in the Era of AI

Inception

I have been teaching Econ5821 since 2022, when I returned to Hong Kong. The contents of Econ5821 evolved over 10+ years. When I just joined CUHK, there was students’ request in department board meetings that the department should offer more courses to Ph.D.s about programming. Another junior colleague and I volunteered to launch a new one in the spring semester. It was called “Computational Methods in Economics.” It was co-taught for many years until I moved to Georgia Tech. The half (six weeks) from me consisted of programming, computing, and machine learning. My colleague taught dynamic programming in macroeconomics and structural labor. It was a bit ironic that the two instructors didn’t unify the programming languages—mine was R and hers was Matlab.

In 2022 my department’s master of science program had been running for almost a decade. The trend of technological development was unmistakenable: big data and machine learning (The AI tsunami was not there yet). To meet students demand, I re-modeled the previous Ph.D. course into a master-level one, with substantial reduction in the academic contents—those methods only useful in writing research papers. It became more accessible for students with economics and finance background, many of whom had insufficient exposure to coding.

There was an interesting twist about my interaction with students. In its initial offering, I thought that I invested a lot of time into this course. Surprisingly, the teaching evaluation was a miserable. I reflected the contents and the way of teaching, and painfully jettisoned some beloved technical materials. The treatment was immediately effective. The next year, I obtained a nearly perfect teaching evaluation. I was proud and bragged about this episode in front of my colleagues in my department’s annual retreat.

Here comes ChatGPT

November 2022 will go down into history. This month, OpenAI launched ChatGPT, and it became the most downloaded app instantly. It was able to answer questions, compose poems, draft emails, and write code. It was the most important technological breakthrough so far in this century.

AI would change the landscape of higher education permanently. Some majors, fields, experts, and professors were horrified. University administration was never prepared for such an impact. My employer announced 4 modes of AI adoption, from the most restrictive to no restriction, as Approach 1 to Approach 4. With zero hesitation, Econ5821 adopted Approach 4: “Use is freely permitted without acknowledgement.”

Econ5821 was special in its code-heavy contents, which was intimidating for some students of economics and finance background. The advent of a chatbot that could write code was immediately seen as a positive aide. I kept an open attitude toward AI in its inception. AI has been part of the toolkit since the 2023 spring revamp.

At the beginning, AI’s capacity was limited. Hallucinations popped up as daily jokes in social media. Econ5821 had a middle term exam at that time, with closed-end data collection, data processing, and statistical analysis tasks. I tested the questions in ChatGPT, and I was convinced that AI was unable to do it.

While we laughed at AI as “too young, to simple, sometimes naive,” AI has been progressing at unprecedented speed, thanks to the scaling law. In 2024, it could handle half of the midterm questions. In 2025, I tested it again with the previous years’ midterm questions, and found the midterm meaningless—I cancelled it.

Three years later, no one doubts that AI has a unique edge and has already revolutionized the way people write code. If you have read this post up to this point, you may reasonably wonder “Why do I still want to enroll in this course?” “Is it a waste of time for me to learn data science?”

I am still teaching it, and I believe it is not a waste of time. As a master’s course, it is an update and enhancement of the economics / finance courses that you learned in your undergraduate study. No matter where you took your undergraduate education, China or USA or somewhere else, undergraduate curriculums are hopelessly outdated. Data science is a direction of economics / finance higher learning that is close to real-world productivity. It exposes you to modern tech toolkit and workflow. It is not about abstract theory, but empirical facts.

In the past, data science had a high wall and deep moat. With the assistance of AI, the entry barrier has been substantially lowered. I envision that in the near future, perhaps 5 years or so, data science becomes part of general education, along with literacy in language, math, and science. At the end of the day, data science is one way to understand the world from a different perspective—the inductive reasoning, in contrast to the deductive reasoning as in standard math and physics.

Even if AI is commonplace, humans must be able to supervise AI and monitor AI. That is, we verify if AI produces what humans intend to produce, and we need to know how to fix issues. If one day AI is so capable that data science is completely automated, humans still need to understand the consequences and implications from AI’s data analysis to make their decisions. Today’s AI is based on large language models, while humans are in the world model. Assuming AI cannot reverse the arrow of time, a human only lives once.

Higher Education

Last week, we witnessed a substantial drop of stock prices of many American software companies. Media attributed it to Anthropic, the company behind Claude Code. Claude Code is the leading AI enterprise of coding agents and assistants. The progress of AI’s coding capacity improves so rapidly that now it starts to trigger an existential crisis of giants such as Adobe, Figma, and Salesforce. The shock spread over to data providers such as Bloomberg and LSEG.

No decent universities are listed in the stock markets, and thus the values of universities are not instantly reflected by investors vote by money. Since the inception of AI, unfortunately, the values of universities all overall the world are declining. It has exposed the sad fact that many skills that are preserved and handed down in the current higher education system indeed have little intrinsic value.

For example, one of the most hated institutions in Chinese Mainland universities is that every student must complete a thesis in order to obtain a degree. A thesis—bachelor’s, master’s, or Ph.D.’s—comes from “research.” However, we can reasonably expect that 95% of undergraduates have no interest and talent for genuine research, and 98% of the theses are a waste of time and printing papers. One of the immediate benefits of AI is that it liberates undergraduates from conducting useless research. AI saves time.

As a university teacher myself, I surely believe and cherish the virtue of higher education. It passes knowledge from Newton, Gauss, and Shannon to the next generation, and contributes new findings and discovery day by day. However, the current system must be revolutionized in view of the impact of automation. The golden days of global higher education are over.