Evaluating Source Authority, Relevance, and Evidence Limits

Good to see you again. Last lesson established that a claim and its evidentiary status are separate: a technical fact, a documented event, a philosophical argument, a myth, and a fictional extrapolation each require different standards. This lesson addresses the next question: what can a particular source genuinely support?

For The Digital World, this matters constantly. A protocol standard can establish what engineers have specified; it cannot prove that every network obeys the specification. An investigative report can document a company’s conduct; it may not settle a broad causal theory about society. A recorded spiritual narrative can be immensely valuable for understanding a particular community’s symbolic world; it cannot authorize a generalized claim about “African belief.” Your task is not to sort sources into “trustworthy” and “untrustworthy” once and for all. It is to judge their authority, relevance, and evidentiary limits for a precise claim.

---

Three questions before you use any source

When you encounter a compelling source, pause before extracting an idea for the Story Bible. Ask three distinct questions.

1. Authority: Why is this person or institution qualified to make this particular kind of claim?
2. Relevance: Does the source address the exact question, setting, period, and scale that matter for your project?
3. Evidentiary limits: What does the source not establish, even if it is accurate and authoritative within its proper scope?

A fourth question follows naturally:

4. Value: Given its strengths and limits, what is this source useful for in your research or fiction?

The important phrase is for this particular claim. Authority is not a universal prestige score.

• An Internet Engineering Task Force standard has high authority for the intended behavior of a protocol.
• A packet capture from a real network has direct authority for what happened in that observed connection at that time.
• A cloud provider’s documentation has authority for what the provider says its service is designed to do.
• A peer-reviewed study may be strong evidence for the results produced by its stated method and sample.
• A journalist may be authoritative about a documented event because they assembled interviews, records, and corroboration.
• A tradition bearer, performer, or community member may be uniquely authoritative about a particular living practice, experience, or interpretation.
• A novelist may have no evidentiary authority about a technical mechanism, yet enormous value as a model for scene structure, estrangement, or tone.

None of those sources is simply “best.” Each has a proper jurisdiction.

A useful distinction:

Term	Core question	Common mistake
Authority	Is this source competent and appropriately positioned to speak on this claim?	Treating fame, institutional branding, or credentials as proof of every statement
Reliability	Does its process tend to produce dependable information?	Assuming a reliable outlet cannot make errors
Relevance	Does it fit your actual research question?	Using a source because its keywords sound related
Evidentiary limit	How far can its evidence reasonably take the conclusion?	Making a source prove more than it can
Value	What can this source contribute to this project?	Discarding a limited source that could still be useful

A strong source can be irrelevant. A relevant source can be weak. A limited source can still be valuable.

For example, a 2018 article about Facebook’s advertising tools may be relevant to the historical development of targeted advertising, but it may be too old to describe the platform’s current product design. A company’s own privacy policy may be authoritative evidence of what it publicly promises, but it is not independent proof that its internal practices always match that policy.

---

Read laterally: investigate the source beyond its own page

Evaluating a source “vertically” means staying on its page, reading its claims, credentials, branding, and links. That is necessary, but insufficient. The source controls what it says about itself.

Lateral reading means leaving the page to investigate the author, institution, publication process, corrections, funding, and independent coverage. You are not trying to uncover a hidden motive behind every sentence. You are establishing the conditions under which the source was produced.

For any source, build a compact provenance record:

• Who created it? Name the individual, organization, editorial body, research group, or community context.
• What is their role? Engineer, standards body, scholar, participant, journalist, advocate, company, government agency, artist, religious leader, archivist?
• What process produced it? Measurement, peer review, interviewing, archival work, oral performance, policy drafting, product marketing, commentary?
• What is its declared purpose? Informing, specifying, persuading, regulating, selling, preserving, interpreting, entertaining?
• What incentives or constraints matter? Funding, deadlines, institutional mission, political commitments, access to sources, audience expectations, legal risk?
• How are errors addressed? Errata, corrections, replies, retractions, revisions, editorial standards, public criticism, version history?

Perspective is not automatically deception. Every source has a standpoint. A security engineer’s incident report, an advertising company’s market analysis, an activist’s testimony, and a scholarly ethnography may all offer something important precisely because they occupy different positions. The question is how that position shapes what they can see, what they may omit, and what claims they can support.

---

Follow the claim back to its strongest available context

Information often becomes thinner as it travels. A study may become an advocacy report; the report may become a news article; the article may become a viral post; the post may become a confident claim in a story discussion. At each stage, qualifiers, uncertainty ranges, and methodological details can disappear.

“Original” does not always mean “true,” but it usually lets you see the claim with its methods, scope, caveats, and uncertainty intact.

For example:

• A press release announcing an AI product is a primary source for what the company claims to be releasing.
• It is not independent evidence that the product works as claimed.
• A research paper is a primary source for its authors’ methods, dataset, analysis, and reported results.
• It is not automatically proof that its conclusion generalizes beyond the studied population or setting.
• A firsthand oral account is a primary source for that speaker’s account and perspective.
• It does not automatically represent every member of a culture, religion, region, or generation.

This is the central discipline of evidentiary limits: do not demand less from a source than it offers, but do not claim more on its behalf than it warrants.

---

Four source types you will use throughout the course

1. Technical sources: standards, documentation, measurements, and incident reports

Technical research often feels unusually solid because it includes exact language, diagrams, code, and formal specifications. Yet technical sources answer different questions.

An Internet standard is usually normative: it says how a protocol or system is intended to behave. It may use terms such as “MUST,” “SHOULD,” and “MAY” to express requirements. A packet trace is descriptive: it records the behavior visible in a particular capture. A textbook synthesizes established knowledge for learning. Vendor documentation describes a product, often from the vendor’s own perspective.

The RFC series is an ideal early example. “RFC” does not simply mean “official Internet rule.” Its stream, status, update history, and errata determine its authority for a particular technical question.

When using a technical source, ask:

• Is it specifying intended behavior, describing observed behavior, or explaining a concept?
• Is it current, updated, obsolete, or historic?
• Is the mechanism presented in a controlled example, a real deployment, or a vendor’s idealized architecture?
• Does it describe a universal Internet property, a particular protocol, or one company’s implementation?
• What conditions does it assume?

Digital World translation: An Internet Standard can become a widely recognized law or treaty among network cities. But it should not imply that every settlement follows the law perfectly. Older RFCs, incompatible implementations, and local workarounds provide technically grounded sources of conflict.

2. Scholarly sources: methods matter as much as conclusions

Peer review is a quality-control process, not a truth machine. It means a work has passed some scholarly screening; it does not guarantee that the study’s sample is representative, its measures are adequate, its interpretation is uncontested, or later work will not revise it.

For a paper such as Generative Agents, later in the course, you will ask:

• What exactly did the authors build and test?
• What population, environment, tasks, and measures were used?
• Does a simulation demonstrate human-like behavior, consciousness, or long-term real-world autonomy?
• Which claims are experimental findings, and which are the authors’ interpretations?
• What would a replication, competing model, or changed task reveal?

A paper can support: “Under these stated conditions, the researchers observed this result.” It cannot automatically support: “Therefore this technology will behave identically in society.”

3. Journalism: evidence for events, not a substitute for every underlying record

Journalism is especially valuable for documented events: corporate decisions, regulatory actions, leaks, lawsuits, interviews, labor conditions, and the public consequences of technology.

A well-reported article may combine documents, named sources, corroborating witnesses, data analysis, and editorial review. Its reliability is strengthened when it distinguishes what it knows from what it cannot verify, links to records where possible, and publishes corrections visibly.

Still, journalism has limits:

• Breaking news is often provisional.
• Anonymous sources can be necessary but difficult for readers to assess.
• Headlines can simplify an article’s actual uncertainty.
• Opinion, analysis, and straight reporting have different purposes.
• One investigation may reveal an important case without proving a universal pattern.

For the project, treat a report on a data broker as evidence about the broker’s documented practices, business model, or regulatory case. Do not silently expand it into “all data brokers do this” without broader evidence.

Digital World translation: Journalism can become the testimony, court record, rumor bulletin, or investigative archive through which ordinary inhabitants learn about hidden institutions. Its partiality is narratively useful: people in the Digital World will often have incomplete, contested knowledge of the powers governing them.

4. Folkloric, religious, and cultural sources: context is part of the evidence

This project will later draw on Nigerian materials concerning spirit, personhood, power, media, wealth, and doubling. The research standard here must be particularly careful.

A cultural source may include an oral narrative, ritual account, film, sermon, ethnography, archive, translation, interview, or scholarly interpretation. It is not a single category called “folklore,” and it does not speak for an entire continent or even a whole nation.

For each source, identify:

• Community and location: Yoruba, Igbo, Edo, Niger Delta, or another named context; rural, urban, diasporic, religious, linguistic, and historical settings may matter.
• Source position: Who is speaking, recording, translating, filming, interpreting, or publishing?
• Genre and purpose: Is this a ritual account, a moral tale, a sermon, commercial film, colonial document, ethnography, or contemporary commentary?
• Historical conditions: How might colonial rule, Christianity, Islam, migration, media technologies, or urbanization shape the material?
• Scope: Does the source discuss one community, a particular tradition, a particular performance, or a broader comparison?
• Interpretive boundary: Is it documenting a practice, analyzing its meaning, or making a claim about literal supernatural causation?

A film such as Living in Bondage can be a rich source for studying popular narrative, spiritual economy, mediated modernity, and social anxiety. It is not a transparent ethnographic report on what all Nigerians believe or practice. A scholarly analysis of a tradition can illuminate historical and social context, while still remaining an interpretation that other scholars or community members may challenge.

This does not make such materials “less real” than technical sources. Their evidentiary value is different. A ritual, story, or film may be precisely the evidence needed to study symbolic meaning, cultural memory, or social imagination.

---

OPVL: a compact tool, not a mechanical score

The following OPVL framework offers a useful way to record an appraisal.

Use OPVL as four connected notes:

OPVL element	Question for your notebook	Example: a cloud provider’s architecture guide
Origin	Who made it, when, and through what institution or process?	Written by the provider to document its own services
Purpose	Why was it made, and for whom?	To explain and encourage correct use of the provider’s platform
Value	What can it support well?	Service terminology, stated design, recommended deployment patterns
Limitation	What can it not settle by itself?	Independent security, cost, labor conditions, market-wide practice, or actual performance in every deployment

One adjustment to the infographic is important: “limitation” is not identical to “unreliable.” Every source has limitations. A router log has limited scope but can be exceptionally reliable for the event it records. A philosophical essay may have no empirical authority, yet be indispensable when deciding what personhood should mean in your fictional society.

---

The source appraisal card

Add this structure to your research notebook. It will later become the foundation of your research database.

Source: Full title, creator, date, and source type.
Claim examined: One precise sentence the source is being used to support.
Origin and purpose: Who produced it, through what process, and for what audience?
Authority for this claim: Why is this source positioned to know, specify, observe, or interpret this subject?
Relevant scope: What place, period, population, system, or tradition does it actually address?
Evidence offered: Data, method, records, technical specification, testimony, textual interpretation, or argument.
Evidentiary limits: What does the source not establish?
Corroboration needed: What other kind of source would strengthen or challenge the claim?
Fictional use: What mechanism, institution, atmosphere, conflict, or metaphor might it inspire?
Status label: Fact, documented event, academic theory, philosophical argument, hypothesis, speculation, myth, or fictional invention.

Here is a model appraisal:

Source: An IETF Standards Track RFC on a protocol.
Claim examined: “The protocol requires participants to follow a specified sequence during connection setup.”
Authority for this claim: High, because the RFC records the agreed technical specification.
Relevant scope: Implementations of that protocol version, not every networked application.
Evidentiary limits: It does not show that all software implements the standard correctly, nor what users experience during failure.
Corroboration needed: Implementation documentation, security research, packet traces, or incident reports.
Fictional use: The connection sequence becomes a border ritual whose legal authority is widely recognized but unevenly enforced.
Status label: Established technical mechanism, subject to version and implementation conditions.

---

Key takeaways

• A source is not globally authoritative; it is authoritative for particular claims within a particular scope.
• Evaluate authority through expertise, provenance, method, institutional process, and error correction.
• Evaluate relevance through fit with the exact mechanism, event, period, population, region, or tradition you are studying.
• Record evidentiary limits explicitly. A source can be accurate without proving the larger conclusion you want to draw.
• Technical standards specify intended behavior; measurements and logs describe observed behavior.
• Peer review is valuable screening, not a guarantee of universal truth.
• Journalism is crucial for documented events, especially when methods, evidence, and corrections are visible.
• Cultural, folkloric, and religious sources require contextual reading. Their symbolic and historical value must not be converted into sweeping generalizations or empirical claims they cannot support.

Next, you will turn these judgments into a usable research database: tagging sources by domain, claim type, reliability, fictional use, and relationships to other sources.