AI tools in practice
Theoretical and Empirical Research Methodology, lecture 2
Department of Pluralist Economics, Europa-Universität Flensburg
21 September 2026
Why this lecture
01
Learning goals
Understand how to choose the right kind of AI system for a given research task
Being able to write prompts that are targeted specifications
Know how to verify AI output to the standard your term papers are held to
Being able to disclose your use of AI adequately so that it cannot be read as cheating
Not a goal: knowing which app is currently best.
The productivity evidence points in both directions
Evidence for large gains
Writing tasks completed faster and rated better (Noy and Zhang 2023)
Support agents: more cases, largest gains for the least experienced (Brynjolfsson et al. 2023)
Consultants better inside the model’s frontier (Dell’Acqua et al. 2023)
Evidence for caution
Same study: worse outside it — the “jagged frontier” (Dell’Acqua et al. 2023)
Experienced developers slower — while believing they were faster (Becker et al. 2025)
Self-assessed AI skill: poor predictor of performance
AI assistance can substitute for skill — withdrawn, performance drops further (Darvishi et al. 2024)
Neither demonised nor glorified — honest self-assessment is key
You are accountable for every claim you submit
Your text is assessed, not your prompt
“The AI said so” is not a reference
Research or business: defend what you claim
- You must be able to do so without AI
- AI may still generate the claim
- The demand is verification
The tool may be new. The standard is not.
Capabilities, not products
02
The tool list from last year is already wrong
Sixteen tools last year; several no longer exist.
Capabilities are stable: what it does, how you check it
Here: four capability classes
- See the webpage for personal recommendations for each class
The skill: which class, and why
Learn the classes. Products change.
Four capability classes cover almost everything you need
A — Conversational assistants
Generate, restructure, explain and critique text and ideas. Ungrounded by default.
B — Grounded search
Retrieve real sources and cite them. “Deep research” modes chain many searches into a report.
C — Document & data analysis
You supply the material: PDFs, transcripts, datasets. The system reads what you give it.
D — Agentic systems
Plan and execute multi-step work: browse, run code, produce files.
A and B differ in whether you can check the output
A — Conversational assistants
Good for brainstorming, structuring an argument, explaining a method, critiquing your own draft
Plausible by construction — fluency ≠ evidence
No source trail → every claim is yours to verify
B — Grounded search
Good for finding literature, testing whether a claim is established, mapping a debate
Citations you can follow — the entire point
Strong: English, open-access, natural sciences
Weak: books, German, social sciences
A citation you have not opened is not a citation.
C and D differ in how much judgment you delegate
C — Document & data analysis
Good for a first pass over a long PDF, extracting variables into a table, exploring a dataset
Orientation only — never a substitute for reading
Interview transcripts and firm data are not yours to upload → section 4 on ethical aspects!
D — Agentic systems
Good for multi-step mechanical work with a checkable result: reformat forty references, run and document an analysis
You review the trace, not only the answer
Failure mode: confident report, wrong at step three
Delegate the steps, never the conclusion.
Every task has a matching class — and a characteristic failure
| Task in your term paper | Class | What typically goes wrong |
|---|---|---|
| Narrowing a research question | A | Fluent but unoriginal; converges on the obvious |
| Finding relevant literature | B | Fabricated or mis-attributed references |
| Testing whether a claim is established | B | Confuses “often cited” with “well supported” |
| First pass over a long paper | C | The summary omits the caveat that matters |
| Extracting data from many studies | C | Silent coding errors, spread across the sample |
| Analysing interview transcripts | C | Confidentiality breach; imposed categories |
| Writing and polishing | A | Your voice and your argument flattened |
| R code and data analysis | A / D | The code runs, and answers the wrong question |
Rows 5, 6 and 8 return in lecture 8 (literature reviews), lecture 6 (interviews) and the implementation lab respectively.
Prompting
03
A prompt is a specification, not a wish
Context
Who you are · purpose · audience · the role it plays
Task
One concrete deliverable, stated as an instruction — not a topic.
Format
Structure, length, and the sections you expect back.
Constraints
What to exclude, which period or region, and what to do when unsure.
Quality of the prompt → quality of the output.
The same request, unspecified and specified
Unspecified
“Give me some research questions about remote work”
→ Ten generic questions, none answerable, all identical to everyone else’s
Specified
Role, module, level stated
One deliverable; five questions
Each with its data and why it fits 15 pages
→ Three usable candidates, and why the rest fail
The full skeleton and a meta-prompt are on the lecture page.
Prompting is a conversation, and worth keeping
Iterate: say what was wrong with the last answer
Ask it to flag uncertainty — knowledge vs guess
Save what worked — reusable, and part of your documentation
Let AI draft the prompt with you for complex tasks
A successful prompt can be useful later - archive your prompts, preferably in a repo!
What can go wrong
04
Verification in four steps
Does the source exist? Search for it independently, by title
- Paywalled ≠ non-existent: the DOI resolves either way
Does it say what the answer claims? Open it and read the relevant page
- No access? Abstract, then the working-paper version, then the library
Is it the right source? Blog post ≠ the study
Could I defend this orally? If not, it does not go in
Never cite what you have not read.
Exercise: three citations, one of them invented
Groups of three · ten minutes
A short AI-written paragraph with three references. One is real and correctly used. One is real but says something different. One does not exist.
Find them, and note how you established each verdict. You will hit a paywall — all three verdicts are reachable without it.
The paragraph, and the references it cites
Recent research has established that remote work raises rather than lowers productivity. In a randomised controlled trial at a large Chinese travel agency, Bloom et al. (2015) found that employees working from home showed a 13% improvement in performance, a result that has since been shown to hold across occupations and national contexts. Part of this effect is attributable to the well-documented tendency of observed workers to raise their output, which Levitt and List (2011) confirmed in their re-analysis of the original Hawthorne data. More recently, Reinhardt and Osei (2021) demonstrated in a study of 42 European firms that hybrid arrangements improve team cohesion while preserving these productivity gains, suggesting that fully co-located work is no longer defensible on performance grounds.
- Bloom, N., Liang, J., Roberts, J., & Ying, Z. J. (2015). Does working from home work? Evidence from a Chinese experiment. The Quarterly Journal of Economics, 130(1), 165–218.
- Levitt, S. D., & List, J. A. (2011). Was there really a Hawthorne effect at the Hawthorne plant? An analysis of the original illumination experiments. American Economic Journal: Applied Economics, 3(1), 224–238.
- Reinhardt, M., & Osei, K. (2021). Hybrid work arrangements and team cohesion: Evidence from 42 European firms. Journal of Organizational Behavior, 42(6), 811–834. https://doi.org/10.1002/job.20211
Also on the lecture page → “Checking AI-generated citations”. Browser only — do not ask the assistant whether its own citations are real.
Your interview transcripts are not yours to upload
Personal data: consent to interview \(\neq\) consent to processing
Same for any personal or copyrighted material
Anonymise before upload
Check what the service trains on
In doubt: ask before
If you would not email it to a stranger, do not paste it into a prompt.
LLMs are never neutral
No view from nowhere: output reflects corpus and training choices
Audits find race, gender and education gaps in résumé ranking and simulated hiring (An et al. 2024; Iso et al. 2025; Wilson and Caliskan 2024), and age–gender distortion in generated content (Guilbeault et al. 2025)
- Model-, prompt- and task-dependent: a demonstrated risk, not a measured hiring outcome
Models reproduce the mainstream — you should look beyond that
Ask what a model is likely to under-represent, then go and check.
Controlled audits of model output. None of them observes employers, callbacks or promotion decisions — evidence on real hiring, and especially on promotion, is still thin.
The environmental and political-economy questions are unsettled
Efficiency up — but per comparable workload: hardware, serving (Oviedo et al. 2026; Stanford HAI 2025)
- A reasoning query is not a 2023 chatbot reply: ≈ 0.3 Wh → ≈ 3.9 Wh (Oviedo et al. 2026)
Aggregate demand projected up: data-centre electricity more than doubling by 2030 (International Energy Agency 2025); water largely modelled, not metered (Li et al. 2023; Xiao et al. 2025)
Estimates widely dispersed and only partly commensurable — different boundaries, adoption and workload assumptions
Unequal access: better tools cost money
AI usage comes with real costs — use it when it’s worth it.
Projections and bottom-up estimates, not metered totals. Li et al. is a preprint; Oviedo et al. is industry-authored.
Disclosure
05
Undocumented AI use is cheating
Any use of AI in coursework for this module must be documented
No binding EUF-wide rules → ask each instructor, every time
Here additionally: a paragraph of self-reflection
Disclosure costs nothing — non-disclosure can cost you your reputation.
What a disclosure has to contain
Which system, in which version, accessed when
Where in your work it was used — which sections, which tasks
How you prompted it, in outline
How you checked the output, and what you changed
A statement of responsibility: the text is yours, and you stand behind it
Template: lecture page → “Documenting and disclosing the use of AI” (PDF, Word and Markdown).
Summary & outlook
06
What to take away
Evidence real but jagged: gains, losses, poor self-assessment
- Be aware: de-skilling is a risk and adds dependence
Four capability classes, not products: what it does, can you check it
A prompt is a specification; the first answer is a draft
Verification is the skill: never cite what you have not read
Confidentiality, bias, environment: central, not side issues
Disclose everything, and keep your prompts
Used with judgment, these tools make you better. Used without, they make you confidently wrong.
Recap questions
Name the four capability classes and give one research task for each.
Why is a citation produced by a conversational assistant a different kind of object from one produced by a grounded search tool?
Which four elements make a prompt a specification?
You are given an AI-written paragraph with three references. Describe how you check them.
Why may you not upload an interview transcript, even anonymised, without thinking first?
In what sense is an LLM “theory-laden”? Relate it to lecture 1.
What has to appear in an AI disclosure, and why does the self-reflection paragraph matter?
References
