The amount of information we have access to is overwhelming, and, consequently, determining whether or not that information is reliable isn't always straightforward. The resources here have been curated to help you evaluate sources as you move through complex information ecosystems (especially online). The usefulness of a source and how “good” or “bad” it is will be determined by your needs and its relevance to your question. Sometimes, you will need scholarly articles that have been peer-reviewed by experts in the field. Other times, you might need first hand accounts from people with lived experiences in your topic of interest. Remember, any kind of resource can be appropriate and useful for your research, as long as you understand the particularities of each source type, as well as their perspectives and biases.
Below is a helpful video from Portland Community College on evaluating sources and finding quality research that will be useful for your assignment:
Video: Evaluating Sources to Find Quality Research. PCC Library (2016).
See below for additional resources on understanding and evaluating sources:
The goal of peer-review in academic publishing is to assess the quality of articles submitted for publication in a scholarly journal. Peer-review is exactly what it sounds like—articles must undergo a process of review by scholars in the field before they can be published in a peer-reviewed, academic journal. Here is an outline of the peer-review process:
Some databases (such as OneSearch@IU) allow you to filter results by peer-reviewed articles. It good practice to double check the journals from which the articles come to make sure that it is peer-reviewed (if this is what you are looking for). You can do this by googling the name of the journal. You can also refer to the following image when trying to determine if an article is peer-reviewed:
Image: Evaluating Information Sources: What Is A Peer-Reviewed Article? Lloyd Sealy Library, John Jay College of Criminal Justice (2019).
It is important to note that peer review is not equivalent to objectivity. Bias and subjectivity can show up in a number of ways, both before and during any process of peer or editorial review. The peer review process (as well as other scholarly mechanisms) also privileges certain ways of thinking, communicating, and knowing, and not all thinkers or communities have access to (or choose to take part in) this system. While peer review is an integral evaluative process within many fields, and is one pathway through which knowledge can be created and shared, we suggest an expansive framework for finding, using, and evaluating sources, and for students and scholars to consider information sources beyond what makes it into the scholarly conversation through peer review when possible and appropriate.
Scholarly sources are intended for academic use—often, these are often the peer-reviewed articles we discussed above. They utilize a specialized vocabulary and extensive citations. Scholarly sources help answer the "so what?" questions and make connections between variables (or issues). Scholarly sources:
Image: Academic Journals, Raising Your Scholarly Profile. Duquesne University (2023).
To learn more about scholarly, popular, and trade sources, explore the links and video below:
Video: Scholarly vs. Popular Periodicals. Vanderbilt University (2017).
Popular sources are intended for the general public and are typically written to entertain, inform, and/or persuade. Popular sources help you answer "who, what, where, and when" questions and can range from research-oriented to propaganda-focused. Popular sources:
Image: Research Process: A Step-by-Step Approach: Find Information Sources: Popular vs. Scholarly. Nash Library and Student Learning Commons, Gannon University.
Browse the databases below to access a variety of popular sources. If you don't see your publication of interest on this list, try searching the library website or contacting a librarian for assistance.
Trade sources (also known as Trade Publications) share general news, trends, and opinions in a certain industry. They are not considered scholarly, because, although they are generally written by experts, they do not focus on advanced research and are not peer-reviewed. Trade publications:
Image: Trade Publication Advertising, Rapp Advertising, Inc.
Many academic books will be edited by an expert or a group of experts. Unlike a scholarly article, which usually focuses on the results of one research project, a book is likely to include an overview of research or issues related to its topic.
Conference proceedings are compilations of papers, research, and information presented at conferences. Occasionally, proceedings are peer-reviewed. Proceedings are more commonly encountered (via databases and other searching) in science and engineering fields than in the arts and humanities.
The Government Printing Office disseminates information issued by all three branches of the government to federal depository libraries (including IU Libraries). Additionally, government departments publish reports, data, statistics, white papers, consumer information, transcripts of hearings, and more. Some of the information published by government offices is technical and scientific.
Theses and dissertations are the result of an individual student's research while in a graduate program. They are written under the guidance and review of an academic committee but are not considered "peer-reviewed" publications.
Grey literature refers to the wide body of reports, conference proceedings, preprint literature, working papers and drafts, personal communications, technical notes, theses & dissertations, and other ephemeral scientific and research materials published by business, governmental, or academic organizations. While not often utilized for student assignments, this literature can be helpful for engaging with new ideas, alternative perspectives, and emerging scholarship.
Adapted from: Understanding & Evaluating Sources, NM State University Library (2022).
Now that we've discussed peer review and other types of sources, let's look at ways to evaluate these sources. Below, we will discuss media timescales, popular & media sources, what to do with sources, and some tests you can perform to evaluate credibility.
Peer-reviewed sources are only one type of source. Let's think about some ways to identify sources and whether or not they will be valuable for your research. Previously, we discussed the fact that sources go through various levels of review. The following graphic helps illustrate the speed at which media is published:
Image: The Media Timescale. Noah Brier, Why is this interesting? Substack (2022).
A source with a slow media timescale doesn't necessarily indicate this it is better or more useful than a source with a fast media timescale. Some research questions require a combination of all three timescales while others—say literature reviews or papers with peer-reviewed sources as a requirement—will require slower timescales. Thinking about which kind of sources you are looking for can help you determine if a source will bolster or detract from your paper.
Lateral reading is a technique, often used by professional fact-checkers, that evaluates a source's credibility through examining the external sources which refer to the original source. Lateral reading involves using Wikipedia, credible news sources, and other references to better understand a source's credibility, funding, reputation, conflicts, and biases. To perform lateral reading, open a new tab in your browser and search the name of your website of interest. You can also search for the author, publisher, and affiliated organization (i.e. funders, board members, supporters, etc.). Explore fact-checking websites such as PolitiFact, Snopes, and FactCheck.org and media bias charts such as Ad Fontes or Allsides. Using lateral reading can help you evaluate many different types of sources including social media, websites, blogs, and news outlets. For more information on lateral reading, see Salem State University's How-To Guide and the Lateral Reading infographic from University of Louisville Libraries below:
Image: Evaluating Online Resources Through Lateral Reading. University of Louisville Libraries (2022).
Video: Sort Fact from Fiction Online with Lateral Reading. Stanford History Education Group (2020).
Popular sources typically refer to general interest publications like newspapers and magazines. While different from scholarly sources, some newspapers and magazines might be useful in helping you answer your research question. For example, if you are performing research on public policy and information campaigns during the COVID-19 pandemic, the Twitter feed of a local health organization might offer valuable information. Magazine articles, newspapers, and blogs can inform you about the public perception of an event during a certain time and how people reacted to said event. Popular sources can also provide simplified descriptions of scholarly research, background information, or offer opinions and more personal points of view on a topic. See our section on News & Newspapers to learn more about evaluating popular sources.
Photograph: Newspaper stand. Ed Yourdon, Flickr (2008).
Below is a method called BEAM, which asks you to consider the function of your source—what it is and how you might want to use it in your paper. For example, a passage from a novel might work better in a body paragraph of your essay rather than in the introduction or conclusion.
Source Function | Explanation | Types of Sources | Where can I use it? |
---|---|---|---|
B: Background | Factual and noncontroversial information, providing context | Encyclopedia articles, overviews in books, statistics, historical facts | Introduction |
E: Exhibit/ Evidence | Data, observations, objects, artifacts, documents that can be analyzed | Text of a novel, field observations, focus group transcriptions, questionnaire data, results of an experiment, interview data (primary sources) | Body/Results |
A: Argument | Critical views from other scholars and commentators; part of the academic conversation | Scholarly articles, books, critical reviews (e.g. literacy criticism), editorials | Body, sometimes in Introduction or in Literature Review |
M: Method | Reference to methods or theories used, usually explicit though may be implicit; approach or research methodology used | Part of books or articles with reference to theorists (e.g. Foucault, Derrida) or theory (e.g. feminism, post-colonialism, new historicism etc.); information on a research methodology | Methods or referenced in Introduction or Body |
Table: Source Functions: Background, Exhibits, Argument, Method (BEAM). UC Merced Library, (2022).
When locating and evaluating resources, you can use the SCAAN test to help determine if they are appropriate to use, reliable, and relevant:
There are a number of other, similar "tests" that you can use to assess the credibility and utility of information and resources you find. These include:
There isn't one perfect way to evaluate a source, and every source has a potential use even if it isn't relevant for your particular research. The frameworks we've provided here are multiple ways to understand the value and use of the sources you'll find, but also consider your own perspective and embodied knowledge when engaging with sources to determine whether and how they should be used.
Whichever evaluation you use, remember that lived experience is a form of expertise and therefore, it is often helpful to consider the position and experiences from which an author writes as part of your source evaluation process. You may also want to reflect on your own biases when reviewing your information. How might your identity and positionality impact your consideration or evaluation of a source? If the source had the opposite position or result, how would that affect your opinion of its validity?
Video: Evaluating Sources. UOW Library (2018).
Artificial intelligence (AI) refers to a set of technologies that enable computers to perform a variety of advanced functions which are typically thought to require human intelligence. These functions include recognizing faces, understanding, interpreting, and creating written and spoken language, analyzing data, driving cars, creating art, and more. AI systems are trained on vast amounts of data, allowing them to identify patterns and relationships which humans may not be able to see. The AI learning process often involves algorithms, which are sets of rules or instructions that guide the AI's analysis and decision-making. Through continuous learning and adaptation, AI systems become increasingly adept at performing tasks, from recognizing images to translating languages and beyond.
Though many people have grown up surrounded by AI technologies that have affected everything from traffic patterns to the products available at the grocery store, with the recent release of ChatGPT, AI has moved into the foreground of our lives. The increased accessibility of AI bring with it a need for AI literacy. In this context, literacy does not simply refer to the ability to use AI technologies but to the combination of knowledge and skills that allow users to critically understand and evaluate AI tools safely and ethically in an increasingly digital world. In our daily lives, we implement information, media, financial, and health literacy when performing all kinds of tasks. When practicing AI literacy, one might ask questions like How does this technology work? What kind of data was this system trained on? What biases are present in this technology? How does this impact my world and the world around me?
This guide, focused on literacy, contains a selection of resources that can help teachers and students navigate AI in classroom settings and can give us all a foundation to ethically and responsibly use AI technologies.
Suggested introductory articles:
Sources: AI Literacy: A Framework to Understand, Evaluate, and Use Emerging Technology (Keun-woo Lee, Kelly Mills, Pati Ruiz, Merijke Coenraad, Judi Fusco, Jeremy Roschelle and Josh Weisgrau; Digital Promise, June 18, 2024); What is Artificial Intelligence (AI)? (Google Cloud); What is AI? Everyone thinks they know but no one can agree. And that’s a problem (Will Douglas Heaven; MIT Technology Review, July 10, 2024); AI Literacy, Explained (Alyson Klein; EducationWeek, May 10, 2023)
Everyday, we utilize different literacies. For example, when we read the news, we might use an information literacy framework to determine whether or not we can trust the media which we encounter. We can ask questions about who created or funded an article, about why the message of the piece is being sent, and about what kind of research went into the piece. Similarly, we can utilize an AI Literacy Framework when utilizing AI-enable technologies or engaging with the outputs of such systems. The following practices, developed by consultants and research at Digital Promise, define how users can understand and evaluate AI-enabled tools and how educators can support AI literacy development.
AI Literacy Practice | Description | Student Looks For |
---|---|---|
Algorithmic Thinking, Abstraction & Decomposition | Develop and/or use a computer’s ability to recognize data and create a prediction or perform an action based on both the situation and stored information without explicit human guidance. |
|
Data Analysis & Inference | Consider the context of datasets, data visualizations, and data collection with criticality. Assess quality of training data for AI tools and leverage AI models and methods to collect, analyze, and visualize data. |
|
Data Privacy & Security | Develop awareness of data privacy and security while fostering ownership and agency of how to protect data in AI-enabled systems. This includes the privacy and security of personal data collected by an AI system or tool and how that data is used. |
|
Digital Communication & Expression |
Understand how AI Systems create synthetic content, evaluate synthetic AI creations, and consider ethical responsibilities when consuming, creating, and sharing AI-enabled products. |
|
Ethics & Impact | Examine the outputs of algorithms and question the biases inherent in the AI systems and tools being used. Consider the benefits and harms of AI tools to the environment, people, or society Importantly, it includes considering how datasets, including their accessibility and representation, reproduce bias in our society. |
|
Information & Mis/Disinformation |
Determine credibility of AI system outputs in digital landscapes. This includes evaluating datasets and AI products/outputs for false, inaccurate or misleading information. |
|
Source: AI Literacy: A Framework to Understand, Evaluate, and Use Emerging Technology (Kelly Mills, Pati Ruiz, Keun-woo Lee, Merijke Coenraad, Judi Fusco, Jeremy Roschelle, & Josh Weisgrau; Digital Promise, June 2024)
AI has many applications at all levels of education. Though we may know AI best for the way it has reshaped student learning (through the proliferation of generative AI technologies that can create text, code, and other types of content), AI is also utilized by teachers and administrators. Predictive AI tools can analyze patterns in student data to forecast outcomes such as graduation rates and student learning milestones. These insights allow educators to intervene proactively but require careful evaluation for potential bias. See some more uses of AI in education below:
Chart: Examples of AI Applications in Education (Digital Promise).
AI also has many potential benefits when implemented in an educational setting. Of course, there are many risks as well:
Graphic: Potential risks and benefits of AI in education (TeachAI).
Below, we have selected frameworks and toolkits that will help teachers and students implement and utilize AI in their classrooms:
Much research has been done on bias in AI. As AI becomes more ubiquitous, it is important to understand that our own unconscious, implicitly biased associations can affect AI models, resulting in biased outputs. Though we might think of technology as neutral, AI has a long history of perpetuating biases present in our society such as racism, ableism, ageism, sexism, homophobia, and more. See below for articles, books, and reports to learn more.
For more books, see the following subject headings in IUCAT: