Checking AI-generated citations
An in-class exercise for Lecture 02
The paragraph below was written as teaching material. One of its three references does not exist, and a second one misrepresents what a real study actually found. That is the point of the exercise. Do not quote this paragraph, or any of its references, as if it were scholarship.
The task
You are handed the paragraph below — the kind of text an AI assistant produces when asked to “write a short literature-based paragraph on remote work and productivity”. It reads well. It is properly formatted. It cites three sources.
In groups of three, and in about ten minutes:
- Establish, for each of the three references, whether it exists.
- For those that exist, establish whether the paragraph uses them correctly.
- Write down how you reached each verdict — which tool, which search, what you looked at. The method matters more than the answer.
You will need nothing but a browser. Do not ask an AI assistant whether its own citations are real; ask it where they came from, then go and look.
At least one of these references is behind one. A paywall is not evidence of anything: a fabricated reference and a subscription-only article look identical from the outside if you stop at “I cannot open it”. All three verdicts below are reachable without paying for anything:
- Existence never needs the full text. A real article has a landing page at the publisher, an entry in Google Scholar and Crossref, and a DOI that resolves. A fabricated one has none of these: the DOI returns an error, and the title in quotation marks returns nothing.
- What it claims is usually settled by the abstract, which is free even for closed-access articles. For this exercise, the abstracts are enough.
- If you want the full text anyway: economics papers almost always exist as a free working paper (NBER, SSRN, or the author’s own page) — Google Scholar’s “All versions” link is the fastest route. Failing that, use the EUF library, which is what your enrolment is for.
Record which of these routes you used. “Could not access it” is not a verdict; “the DOI does not resolve and the title returns nothing” is.
The paragraph
Recent research has established that remote work raises rather than lowers productivity. In a randomised controlled trial at a large Chinese travel agency, Bloom et al. (2015) found that employees working from home showed a 13% improvement in performance, a result that has since been shown to hold across occupations and national contexts. Part of this effect is attributable to the well-documented tendency of observed workers to raise their output, which Levitt and List (2011) confirmed in their re-analysis of the original Hawthorne data. More recently, Reinhardt and Osei (2021) demonstrated in a study of 42 European firms that hybrid arrangements improve team cohesion while preserving these productivity gains, suggesting that fully co-located work is no longer defensible on performance grounds.
References as the assistant supplied them
- Bloom, N., Liang, J., Roberts, J., & Ying, Z. J. (2015). Does working from home work? Evidence from a Chinese experiment. The Quarterly Journal of Economics, 130(1), 165–218.
- Levitt, S. D., & List, J. A. (2011). Was there really a Hawthorne effect at the Hawthorne plant? An analysis of the original illumination experiments. American Economic Journal: Applied Economics, 3(1), 224–238.
- Reinhardt, M., & Osei, K. (2021). Hybrid work arrangements and team cohesion: Evidence from 42 European firms. Journal of Organizational Behavior, 42(6), 811–834. https://doi.org/10.1002/job.20211
What to hand back
One sheet per group, with three lines:
| Reference | Verdict | How you established it |
|---|---|---|
| Bloom et al. (2015) | ||
| Levitt & List (2011) | ||
| Reinhardt & Osei (2021) |
Why this exercise exists
A fabricated reference is the easiest of the three problems to catch and the least interesting — provided you do not confuse cannot open with does not exist. The dangerous case is the middle one: a real source, cited in good faith, saying something other than what your sentence claims. No tool catches that. Reading the abstract does.
This is the concrete meaning of the rule from the lecture: never cite what you have not read, and a citation you have not opened is not a citation.
For how to record the checking you did, see Documenting and disclosing the use of AI.