Can AI actually read the academic literature?

A perspective paper on the limits and benefits of AI for academic research scholarship - part 1

WHY RESEARCHERS WANT AI TO READ THE LITERATURE

I want to start in a very fundamental place - it is the place that all scholars at one time or another find difficult or boring, and that is mapping the field of knowledge in any given research area. The famed, hated and dreaded, literature review. This is a site of massive scholarly psycho-drama, because it takes a lot of time and cognitive energy and can produce profound feelings of insecurity (have I read enough, have I missed something, am I going to be outed as an idiot for a fundamental misunderstanding). And it’s very, very time consuming. In the arena of publish or perish, the temptation for shortcuts abound. So far, so well known. So it should come as no surprise that scholars are excited about the possibility of out-sourcing this annoying task to the AI in their pocket. Why not? Research would be so much better if all that cognitive effort of learning, interpreting, thinking, discovering, uncovering could be bypassed and I could get the bot to do it instead. Presto, here is your lit review of existing scholarship in the field. Now, I can get on telling you what I want to say.

I get it, I do. But I think we need a fundamental reality check of what AI can currently do, and what it cannot do because we need to penetrate the line between the belief of what it does, with what it actually does. Top line. It cannot think for you, because it does not think. It cannot understand, and it cannot judge, it cannot make decisions. Two fundamental areas where AI cannot assist you: reading - and the cognitive magic that happens as you read and create new original thinking and insights - and writing, where you do the same, as you create novel text and iterate your discovery through each draft.

But in this paper I want to begin with part 1: reading and why AI can’t do it for you.

Can AI read all the literature?

Let’s start with the basics. Can AI read all the literature you need to access to produce a novel piece of research? No. Can AI access all the published research? No. That single truth should scare you if you have been using AI to ‘summarise the literature’. This is not a controversial statement, this is not how LLMs work. It does not sit atop Elsevier’s servers reading every paper ever published. The training data is much more limited and what it can access produces considerable intellectual drift.

Any research that is not in the public domain - that is the vast majority of peer reviewed published research - cannot be accessed by AI as they are behind paywalls. So far, the publishing houses - sensing the death of their own industries for which I’m sure many of us won’t shed a tear - have resisted making deals with AI companies to get access yet. So until and unless these deals are made or every piece of scholarship is publicly available (ie gold open access), AI cannot see it. Same with books, or any other original piece of thinking.

AI’s sphere of understanding then does not include Elsevier, Springer, Wiley, Taylor &Francis/Routledge and Cambridge University Press for example. It had access to a limited number of papers in training: arXiv, PubMed, SSRN, institutional repositories, author preprints, and so on. In some disciplines (eg physics) much is already freely available, but this is not the case in humanities and most other subjects. It has therefore a much more limited view than your library subscriptions of original thinking, and more importantly, the map of scholarly knowledge is missing. It’s not the odd byway, it’s more like a couple of continents.

What is it AI ACTUALLY reading?

So what is AI actually reading, if not the vast majority of paywalled peer reviewed papers? Whilst you may have thought it is reading Foucault’s original book for instance, instead AI is reading: publicly available web pages, open access journal articles, books whose licence permits training (not all books), public repositories, government publications, Wikipedia, student essays and other vibes, code repositories and licensed data sets where agreements have been made. Now you can see much of that (gold access apart) is derived from original thinking, but not the thinking itself.

It is reading a paper that cited a paper that cited a paper in an infinite web of citations of that original paper. Suppose paper A is behind a paywall, but 20 related later papers ‘summarise paper A’. AI has never seen paper A, but it has seen 20 summaries of it, so it develops what it would term a ‘fairly accurate representation’. Sit with that for a minute. The first is the assumption that there are 20 papers accurately summarising Paper A. I have never seen 20 published papers merely summarising a previous paper, because obviously that is not original thinking, that would not get published, nor is that the job of 20 later papers. But to the AI, those 20 papers citing that previous papers is constructing ‘fairly accurate’ picture for it of the unseen paper’s real meaning. Worse, if those 20 papers all have different takes (postcolonial, feminist etc) the AI will ‘smooth out’ the contradictions and meaning and include 20 interpretations of the original paper condensed into a single paragraph that somehow has all of them. Clearly this is now nothing like the original paper at all. There is drift and more drift.

Even if it does have access to the original thinking because the model did see the original source during training, it doesn’t retrieve and compare the original with a secondary source when you ask it a question. It generates text from its internal statistical representation. So even access does not guarantee fidelity in the way a human scholar would read and understand it. All this to say, your summaries, are not actual summaries as you a human would have produced from the original material.

Instead of a complete knowledge map, it accesses papers that inexorably drift away from the original position.

CAN AI UNDERSTAND nuanceS IN ACADEMIC PAPERS?

If you ask AI to ‘summarise X Nature paper’ since it cannot access the original it will find the abstract, the author summary (if available) the press release (!), news coverage (!), reviews and combine to give you your answer. You, the user, thinks it is summarising the actual paper. If you put together a whole lit review based on this avalanche of intellectual drift, you are in serious trouble. It doesn’t do nuance. It doesn’t understand that a press release it not an accurate representation of the detailed research of the paper itself. When you are constructing novel thinking, you need detail not top line.

What about if you upload a series of PDFs so you know it now has access to defined data set of original thinking? Sure it can ‘summarise’ that information for you - but how will you know what was novel, what single paragraph you might have read that sparked a new idea?

How will you know - most importantly - what was not there?

Scholars don’t read papers to extract and summarise information, we read to find meaning, connections, and crucially, what is missing. If you are using AI to summarise papers for you, what are you actually doing with the summary? I am genuinely interested. What do you think that summary represents, and how are you going to use it? You’ve now bypassed the part of the process where thinking about problems and solutions was initiated, so what is the summary going to do for you?

I know what you are reaching for: security. You are trying to avoide not only the hard work of actually thinking, but also trying to overcome the terror of thinking you might have an incomplete picture. The irony is not lost. Not only can AI not give you that, it gives you a false belief you have it and that is where the real chasm in thinking lives.

Moreover, AI, in delivering a summary, will generate the most statistically probable continuation based on the patterns they have learned. In areas where scholarship is contested (everywhere!), this often produces a smooth synthesis of competing interpretations rather than preserving their tensions. So imagine asking ‘What is Foucault’s conception of Power?’. There is no consensus - the scholarship might move between productive, discourse, subjectification, geneology, governmentality, biopolitics. These are not errors, they’re interpretations, but the model will produce something that looks like a composite position. More drift. I can’t speak to the sciences, and I assume this is where many of the apps promising summaries of research are being marketed and are potentially useful. I don’t know. You can ask AI to tell you the method in a paper, to summarise results. Is it OK to ask it - what are the risk factors of Type II diabetes? Maybe, but try asking it for Derrida’s understanding of differance. It’s not the same. But the idea that it is the same has taken hold, fast.

AI can often generate a plausible synthesis of the textual evidence it has access to - that is as far as it can go. It is this very plausibility that can fool you into thinking it is doing more than it is. Whether that synthesis faithfully represents the primary literature—or instead reflects the balance, biases, and simplifications of the surrounding discourse—is a separate question that requires human evaluation, but you cannot make that evaluation if you have not read the sources. You don’t know, what you don’t know. And you never will.

I think we can all agree in scholarship, the devil, or novelty, is in the detail. Even the idea of ‘simplyfying complex arguments’ - something most AI app claims make - means reductionist presentation of information, not nuance. Not complexity. If you are relying on compressed text, what got squeezed out?

What can AI Do well

This is not about hating on AI. I use it all the time for various things. You can use AI to accelerate the work around thinking, but not replacing thinking itself and that thinking takes place, not passively reading an (inaccurate) summary of things you should have read, but you reading and then writing about it yourself. Taking your own notes. Writing your own thoughts. And unfortunately the thing you are most likely trying to get relief from - the hard bit, the cognitive friction - is the one thing you still need to do.

So what can you use AI for, research wise:

• Organising your own notes taken from original source material by you, structuring literature notes you've already written, tagging and categorising ideas when you have organised the tags and categories, building searchable knowledge databases, deduplicating notes, creating indexes

• Project plans, thesis timelines, grant timelines, publication workflows, conference schedules, milestone tracking

• visualisation of timelines, conceptual maps, flowcharts, process diagrams, research workflows, decision trees (after you propvide all the intellectual content)

• data organisation providing you have submitted the data, coding interview themes (top level swipe), arranging qualitative excerpts, producing comparison tables, cleaning spreadsheets, identifying duplicates, reformatting datasets (note, not analysing, and not reading for you).

• administrative work in the form of drafting emails, meeting summaries, action lists, project management, remniders, calendar planning, checking consistency (in use of terms for examples in your own documents)

• Repetitive formatting tasks, converting references, reformat tables, convert styles, standardise headings, produce appendices, tidy documents

• learning software applications like hoow to use Nvivo, how to make a graph in R etc

• presentation help, such as slide layouts, diagrams, infographics, teaching visuals, workshop activities

These are all essentially information-processing tasks.

WHAT CAN’T AI DO IN SCHOLARSHIP

AI struggles with novelty, interpretation and judgement. Deciding whether an argument is genuinely original, identifying a paper's strategic contribution to a field, understanding the politics of a discipline, recognising why one journal will reject something another will publish, knowing what reviewers in a specific field currently care about, spotting subtle conceptual weaknesses, advising whether a paper is "good enough" for a target journal, making experienced editorial judgements about positioning, emphasis, and scholarly significance. It cannot do that. That is a skills development issue for you as a scholar.

It cannot answer whether your contribution is actually novel. or whether this approach is the strongest framing for this audience, or what will Reviewer 2 object to first, It can’t tell you whether this paper should become one paper or three, or which journal gives this argument the best chance of acceptance? It cannot think for you. It. cannot bring judgment. These things require more than summarisation. They rely on disciplinary judgement, tacit knowledge, and an understanding of audience and standards that are not fully captured by text alone.

But mainly, it can’t do the hard part. When you substitute AI for cognitive effort, you are disabling the part of the brain that comes up with great ideas. Reading and writing are both component parts of that.

WHY READING RESEARCH PAPERS STILL MATTERS IN THE AGE OF AI

The first fundamental place researchers are turning to AI is for it to read papers for them. Every day I see yet another app making claims that they can do the work for you, and can effectively summarise your papers, produce a literature review and so on, relieving you of this burden. How to use AI, ethically and practically, in research is something as a coach I am repeatedly asked about by clients. Researchers are feeling either left behind as they assume younger colleagues are using AI with ever increasing deftness to produce novel research faster, or younger researchers actually buying into this myth of a productivity goldmine that can simply remove the hard work of being an original thinker. In fact, these scholars are taking a much greater risk, lacking the 20 years of reading behind them that will provide them a more reliable scholarship map.

Researchers often ask me whether AI can save them from the hard part of research. Unfortunately, the hard part is precisely where scholarship happens. The intellectual strain cannot be outsourced. Reading, writing, interpreting, doubting, revising and thinking are not obstacles to originality—they are its conditions. AI can remove some friction around research. It cannot remove the intellectual work that makes research worth doing.

FAQ

Q. Can CHAT GPT read academic journals?

Peer reviewed journals are behind a paywall, so as yet, cannot be accessed by AI models. It can read only open access material.

Q. Can AI access paywalled research

No. Not yet. Companies would need to do individual deals with the publishers, which has not yet happened.

Q. Can AI write a literature review.

No. Because you need to construct a complete map of knowledge and AI cannot provide it. You must do the thinking and material selection.

Q. Is AI accurate for humanities research

No. Because it is about nuanced interpretation, not fact retrieval.

Q. Can AI summarise journal articles

Yes, but extracting facts is not the task of a scholar. What are you doing with that?

Q. Should PhD students use AI for literature reviews?

Absolutely fatal. No. Fastest way to fail a PhD. You literally have no map of the scholarship from which to make judgments, this is actually one of the tests of a PhD. In a defence or viva, you will be asked about the papers you have read…and if you have not read them, BIG RED FLAG. Remember a PhD is an exam. And one element of passing that exam is to have read the papers and be able to prove your indepth understanding of the nuances of that paper.