I’m beginning my journey into Progress Studies by summarising and synthesising some of the literature scattered around.
Here, I show an example of the potential power of Progress Studies. The policy implications of the paper I summarise in this post demonstrate the low-hanging fruit available to the field. If we were to implement the policy prescriptions of the paper below, we could improve the rate of scientific knowledge being created.
Short summary
I summarise “Invisible Geniuses: Could the Knowledge Frontier Advance Faster?” by Ruchir Agarwal and Patrick Gaule (the actual paper is unusually short, concise, and readable for an Economics publication). For an ungated link see here.
- The paper attempts to better understand the determinants of idea/knowledge production.
- There are two key findings:
- Firstly, individuals who are ‘talented’ as teenagers (proxied by International Mathematical Olympiad [IMO] score, a popular international Maths competition) are very capable of advancing the knowledge frontier later in life. On average, the higher the IMO score, the more likely a participant is to: obtain a Maths PhD, obtain a Maths PhD from a top 10 research school, publish more in academic journals, receive more citations, receive notable accolades such as the Fields Medal, or be a speaker at the International Congress of Mathematicians (ICM).
- Secondly, if these capable individuals were born in poorer countries, they are much less likely to contribute to the knowledge frontier than individuals born in wealthier countries. For example, on average IMO participants from low-income countries produce 34% fewer publications and receive 56% fewer citations than IMO participants from richer countries with the same IMO score.
- The findings suggest that one way of developing the knowledge frontier faster is by creating policies to target these low-income students in order to support them in their academic careers. By doing so, we could improve the rate of scientific progress.
I’ll now proceed to write a slightly longer summary of the paper. The structure will stay largely similar to the original paper i.e. follows the same order, but with less detail and excludes the regressions.
Key questions the paper investigates
- How much does talent displayed in teenage years affect the amount of knowledge produced later in life?
- Conditional on a given level of talent in teenage years (IMO score), how much does the country of birth influence the quantity of knowledge produced later in life?
Data
The International Mathematical Olympiad (IMO) is a prestigious mathematical competition held annually since 1959. Individuals from across the world represent their countries and compete to win bronze, silver, and gold medals. Participants confront six questions with different levels of difficulty. Each problem is worth a maximum of seven points and the highest possible score is 42 points. The authors collect data on 4710 IMO participants from 1981-2000. This data contained the year of participant, country of origin, points scored, and type of medal achieved.
The authors also collected data on the long-term outcomes in mathematics for these individuals. To collect data on PhD theses, they used the Mathematics Genealogy Project. This project aims “to compile information on all mathematicians in the world.” This source had information on: the name of the student, the university they attended, the name of the advisor, graduation year, and dissertation topic.
The authors used MathSciNet to collect bibliometric data. This source has information on the total publications and citations broken down by author.
Agarwal and Gaule also collected data on particularly notable/prestigious awards in Mathematics. These were IMO participants who later became speakers at the International Congress of Mathematicians (ICM) or won Fields medals.
Finally, data was collected on the employment of the IMO medalists (2272 of the 4710 participants won medals), to see if they were employed in mathematics academia, outside of mathematics but in academia, in industry, or didn’t have an online profile.
1. How does talent (proxied by IMO score) as a teenager relate to later research success?
Figure 1 (below) plots various mathematical achievements (y-axis) against the number of points scored on the IMO (x-axis).
The first four graphs (three on the top row and one on the bottom-left) show a clear positive relationship between points scored at the IMO and subsequent mathematical success. Thus, on average the more points a participant scores at the IMO, the more likely they are to get a Math PhD, a Math PhD from a top 10 school, more publications (in logs), and more citations (in logs). The other two graphs show the relationship between points scored and exceptional Mathematical achievements later during their research careers (Fields Medal and ICM speakers). These show a positive but weaker relationship than the other graphs.
The authors then regress points scored at the IMO (dependent variable) and subsequent achievement (the six outcomes in Figure 1), whilst controlling for cohort, and country of origin. The regression (see paper) suggests that for each additional point scored at the IMO, there is on average:
- 1 percentage point increase in the likelihood of obtaining a PhD
- 2.6% increase in publications
- 4.3% increase in citations
- 0.1 percentage point increase in the likelihood of becoming an ICM speaker
- 0.03 percentage point increase in the likelihood of becoming a Fields medalist
Additionally, they find that if individuals score more points in the more difficult problems, then this is more predictive of future mathematical achievements than scoring points in the ‘easier’ problems.
Agarwal and Gaule then compare IMO medalists to mathematicians who didn’t participate in the IMO. They constructed a sample of all PhD students obtaining a PhD in Maths, and a subsample of PhD students from top 10 schools. They compared these samples against the accomplishments of bronze, silver, and gold IMO medalists who have a PhD.
Figure 2 (below) shows that for each outcome, the medalists (particularly the gold medalists) outperform PhD students and PhD students from top 10 schools, who did not participate in the IMO. Medalists consistently get more publications, citations, go on to become speakers at the ICM, and receive Fields medals during their careers.
In fact, the probability of receiving a Fields medal is fifty times larger for IMO gold medalists than the corresponding probability for a PhD graduate from a top 10 mathematics programme.
2. Conditional on a given level of talent in teenage years (IMO score), how much does the country of birth influence the quantity of knowledge produced later in life?
For the purpose of statistical analysis, the participants are grouped in terms of their countries income level (high income, upper middle-income, lower middle-income, and low-income). This grouping is a proxy for differences in opportunities and environment. The authors emphasise, “while our regressions explicitly control for IMO scores, it is worth noting that participants from developing countries do not score lower at the IMO than participants from developed countries.”
Figure 3 shows points scored at the IMO in five-point intervals (x-axis) against the share of PhD students receiving a PhD in Maths (y-axis).
Largely we can see that for each number of points scored at the IMO, the high-income countries (black) obtain the most Maths PhDs, followed by upper-middle (green), then lower-middle (grey), and finally low-income countries (blue), who received the least amount of Maths PhDs.
The authors then conduct another regression. This time the dependent variables are successful outcomes: PhD in Maths, PhD in Maths from a top school, publications (log), and citations (log). The main independent variable here is the income group of participants alongside control variables, and importantly the number of IMO points scored. Thus, we’re controlling for ‘talent’ and investigating the effect of the country’s income on future successes in Mathematics.
For the sake of brevity I have left the regression table out. The results of the regression suggest that participants from low and middle-income countries have less research accomplishments in their later careers than those individuals from richer countries with the same IMO score:
“IMO participants from low-income countries are 16 percentage points less likely to do a PhD and 3.2 percentage points less likely to do a PhD in a top school; they produce 34% fewer publications and 56% fewer cites.”
In fact, IMO participants from low-income countries are approximately half as likely to obtain a PhD from a top-tier school than their rich country participants with the same IMO score. This effect holds true of participants from middle-income countries, albeit with a smaller effect size. The IMO participants from low and lower-middle income countries are not more likely to be employed in non-mathematics academic positions, or in industry jobs.
The authors suggest that if IMO participants from low-income countries were producing knowledge at the same rate as those from high-income countries, then they would produce approximately 10% more publications, and 17% more citations.
Conclusion and policy recommendations
Firstly, the paper finds a clear link between ‘talent’ as measured by IMO score, and future research success. Secondly, the paper finds that conditional on a given level of talent, IMO participants from lower-income countries contribute to the research frontier substantially less than richer country participants. That is, despite achieving the same IMO scores, they contribute significantly less research later on in their careers.
Although this paper focused on Maths (the dataset was a really innovative way to overcome many empirical challenges), the same arguments could be applied to other fields. A particularly poignant point is that in Maths, people who have the highest levels of talent as teenagers (gold medalists) are for more likely to do significantly better across a variety of metrics designed to measure successful research compared to other individuals. This talent is rare. Losing this talent seems to suggest a significant waste in terms of scientific progress that could have been made otherwise. The authors suggest that “many geniuses from poor countries are never discovered or given the chance to excel as teenagers in the first place.”
There are a number of policies that could be administered in order to support the individuals from lower-income countries. Firstly, we could give scholarships/fellowships to these students in order to alleviate their financial concerns and support their development. Secondly, elite schools could encourage applications from talented individuals in developing countries. For example, MIT reaches out to international communities of talented individuals and provides them financial support if they need it. Finally, it could be possible to improve the training standards at universities in developing countries to nurture the talented individuals who don’t want to leave their country of birth.
Caveats/Comments
- I left out a number of caveats the authors raise. For example, there are potential competing explanations for the results.
- A couple ideas for future research, perhaps:
- Firstly, we have the results for the Math Olympiad. Can we generalise the results to other subjects? One way of doing so would be to look at the other (less well known) international competitions, such as the Physics Olympiad etc.
- Secondly, this makes a clear case for international comparisons. I wonder how things operate at a within-country level? I.e. are there any similar set-ups, where we could exploit the use of domestic national competitions for example.
- Thirdly, let’s say that it’s hard to obtain (expensive) financial scholarships for students from developing countries to top tier universities such as MIT/Harvard. One alternative solution may be to match students from developing countries with famous Professors from developed countries. Maths research is often an individual pursuit. So it may be possible to emulate the academic environment that leads to later Maths success, by providing access to mentors or access to online classes at top-tier universities.
- It may be able to ‘sell’ this policy to companies/rich individuals. If we have mechanisms enabling us to identify talented individuals from an early age, a company could profit from a smartly administered income-sharing agreement.
Sam Altman recently had an interesting twitter thread that relates to theme of tapping into under-utilised potential:
I’m on Twitter @krisgulati, where I tweet about the causes and consequences of progress, economic growth, technological change, and innovation. I have made a Progress Studies subreddit to foster discussion. You can follow my work on the Progress Studies LinkedIn page or the Progress Studies Facebook page.
If you like this blog and want to see more posts on Progress Studies you can support me (regularly) on Patreon or for one-off donations visit Buy me a Coffee.
You can subscribe here
Progress Studies 