Ch. 01 Thinking Critically About Research

What is “Research” and Why Should I Use It?

Research always begins with the goal of answering a question. In your quest to answer basic research questions, you turn to a variety of different sources for evidence:  reference resources, people, evaluative and opinionated articles, and other sources.  All along the way, you continually evaluate and re-evaluate the credibility of your sources.

For example, if you wanted to find out where you could buy the best computer within your budget, your question might be “what kind of computer should I buy and where should I buy it?” To answer your questions about computers, the first research tool you might use is the phone book, where you would look up “Computer retailers” in the yellow pages.  You might also ask friends where they got their computers and what they thought were the best (and worst) stores to go to.  You would probably also talk to your friends about the kind of computer they bought:  a Windows-based PC versus a Macintosh computer, or a desktop versus a laptop computer, for example.  You could go to a computer store and ask the salespeople for their advice, though you would perhaps be more critical of what they tell you since they are biased.  After all, salespeople are trying to sell you a computer that they sell in their stores, not necessarily the “best” computer for the amount of money you want to spend.  To get the opinions of computer experts, you might do research in computer magazines or web sites, looking for reviews and ratings of different models of computers in your price range.

Of course, you could skip this research process entirely.  You could simply go to a store and buy the first computer in your budget based on nothing more than a “gut feeling” or based on some criteria that has little to do with the quality of the computer—the color, for example.  

Who knows?  By just guessing like this, you might actually end up with a computer as good as you would have ended up with after your research.  After all, researchers can never be certain that the evidence they find to answer their research questions is entirely correct, and the fact that there are different kinds of computers available suggests it is possible for people to look at the research and reach different conclusions about what is the “best computer.”  Talk to loyal Macintosh computer owners and you will get a very different answer about “the best” kind of computer than you will from loyal Windows PC owners!  

Nonetheless, the likelihood is quite high that the computer you bought after careful research is a better choice than the computer you would have bought after conducting no research at all.  Most of us would agree that you have a better chance of being “right” about your choice of computer (and just about anything else) if that choice is informed by research.

Exercise 1.1

Working alone or collaboratively in small groups, answer the following questions:

•    What are some examples of some of the decisions you have made that were based on a research method similar to the one described here?  What do you think would have been the result of your decision had you not done any research?

•    Can you think of any decisions that you have made that were not based on research?  Would these decisions have turned out more favorably had you conducted some basic research?

•    What kinds of decisions do think are potentially best made without research?

What’s Different about Academic Research?

The reasons academics and scholars conduct research are essentially the same as the reasons someone does research on the right computer to buy:  to find information and answers to questions with a method that has a greater chance of being accurate than a guess or a  “gut feeling.”  College professors in a history department, physicians at a medical school, graduate students studying physics, college juniors in a literature class, students in an introductory research writing class—all of these people are members of the academic community, and they all use research to find answers to their questions that have a greater chance of being “right” than making guesses or betting on feelings.

Students in an introductory research writing course are “academics,” the same as college professors?  Generally speaking, yes. You might not think of yourself as being a part of the same group as college professors or graduate students, but when you enter a college classroom, you are joining the academic community in the sense that you are expected to use your research to support your ideas and you are agreeing to the conventions of research within your discipline.  Another way of looking at it:  first-year college students and college professors more or less follow the same “rules” when it comes to making points supported by research and evidence.

A Student Profile:

Daniel Marvins, New to Academic Research

Daniel Marvins is a first year college student at a large public university in the Midwest.  While he certainly wrote plenty of essays when he was in high school, Marvins thought that the kind of research writing his teacher was asking him to do for his writing class was different.  

“In high school, we wrote more about stories and poems and newspaper articles we read,” Marvins said.  “We didn’t do a lot of research, other than looking things up on the web.”

Marvins was ready for the challenge of tackling the thinking and research that would be expected of him in college.  But he still wasn’t sure about being “an academic.”  “I never thought of it that way, because I didn’t really see how the stuff I had to write for school made me anything like my teachers.  But I guess I’m starting to see the connection.”

Read Marvins’ “Working Thesis Essay” in Chapter 5, “The Working Thesis Exercise.”

Primary Research Versus Secondary Research

Before you begin to answer your questions, you’ll need to know about two types of research: primary research and secondary research.  And, you’ll need to learn about the differences between them.

Primary research is usually the “raw stuff” of research—the materials that researchers gather on their own and then analyze in their writing.  For example, primary research would include the following:

•    The experiments done by chemists, physicists, biologists, and other scientists.

•    Researcher-conducted interviews, surveys, polls, or observations.

•    The particular documents or texts (novels, speeches, government documents, and so forth) studied by scholars in fields like English, history, or political science.

Secondary research is usually considered research from texts where one researcher is quoting someone else to make a point.  For example, secondary research would include the following:

•    An article in a scientific journal that reported on the results of someone else’s experiment.

•    A magazine or newspaper account of an interview, survey, or poll done by another researcher.

•    An article in a scholarly journal or a book about a particular novel or speech.

When you quote from another article in your research project, your writing becomes an example of secondary research.  When other researchers quote information from your research project in their  research project, your research project is considered a secondary source for them.  And if a researcher decides to write about you (a biography, for example) and if that researcher examines and quotes from some of the writings you did in college-- like the research project you are working on right now-- then your project would probably be considered a primary source.

Obviously, the divisions between primary and secondary research are not crystal-clear. But even though these differences between primary and secondary research are somewhat abstract, the differences are good ones to keep in mind as you consider what to research and as you conduct your research.  For example, if you were writing a research project on the connection between pharmaceutical advertising and the high cost of prescription drugs, it would be useful and informative to consider the differences between primary research on the subject (an article where the researcher documents statistical connections) and the secondary research (an essay where another researcher summarizes a variety of studies done by others).  

Of course, the term “secondary” research has nothing to do with the quality or value of the research; it just means that to answer the questions of your research project and to support your point, you are relying in great part on the observations and opinions of others.  

Most research projects completed by students in writing classes are based almost exclusively in secondary research because most students in introductory writing classes don’t have the time, resources, or expertise to conduct credible primary research.  However, sometimes some modest primary research is a realistic option.  For example, if you were writing about the dangers of Internet-based computer crime and someone on your campus was an expert in the subject and was available for an interview, your interview of her would be primary research.  If you were writing about the problems of parking on your campus, you might conduct some primary research in the form of observations, surveys of the students that drive and try to park on campus, interviews of the campus officials in charge of parking, and so forth.

Exercise 1.2

Working alone or collaboratively in small groups, answer the following questions:

•    What other sorts of evidence do you think you would find that would count as “primary” research?  What other sorts of evidence do you think would count as “secondary” research?  

•    Think about the kind of topics you are interested in researching and writing about.  What sorts of “primary” research can you imagine examining that might be useful in your writing?  What sorts of “secondary” research can you imagine examining that might be useful in your writing?

Scholarly versus Non-Scholarly Sources

Before you begin to research you should be aware of the difference between “scholarly” and “non-scholarly” or popular sources.  

Scholarly or academic publications are those where academics publish their research and opinions about topics of concern in their discipline.  By and large, scholarly publications are highly specialized periodicals, as many of their titles suggest:  College Composition and Communication, Foodservice Research International, or the Journal of Analytic Social Work.   Scholarly periodicals tend to be published less frequently than popular sources, perhaps monthly, quarterly, or even less often.  For the most part, the readers of scholarly journals are scholars themselves interested in the specific field of the publication—in other words, the articles in these publications are written for academics (both students and teachers) interested in the field, not a “general audience.”  Because of the audience, the language of academic journals is often specialized and potentially difficult to understand for a reader not familiar with the field.  

Scholarly or academic sources tend to be kind of bland in appearance: other than charts, graphs, and illustrations that appear predominantly in scientific publications, most academic journals include few color photos or flashy graphics.  Most academic journals are not published in order to make a profit:  while they frequently include some advertising, they usually only include a few ads to offset publication costs.  Also, most academic journals are associated with academic organizations or institutions that subsidize and support their publication.  Unless you are a subscriber, chances are the only place you will find most of these journals in your college or university library.

Usually, the articles that appear in academic journals indicate where the writer’s evidence comes from with footnotes, end notes, or information in parentheses.  Most academic articles end with a “bibliography” or a “works cited” page, which is a list of the research the writer used in his essay.  This practice—generally called “citation”—is particularly important in scholarly writing because the main audience of these articles (other scholars) is keenly interested in knowing where the writers got their information.  As a member of the academic community, you too will have to follow some system of citation in the research project you do for this and other classes.  

Hyperlink:  See “Chapter 12: Citing Your Researching Using MLA or APA Style.”

Non-scholarly or popular sources tend to be written by journalists and writers who are not necessarily experts about the subject they are writing about.  While there certainly are specialized popular sources, they tend to have names most of us have seen on the magazine racks of grocery and drug stores—GQ, Cosmopolitan, Better Homes and Gardens, Sports Illustrated, and so on—and even specialized popular sources tend to be written with a more general audience in mind. Writers of popular sources reach a general and broad audience by keeping the style of the writing in their articles approachable to people from a variety of different educational backgrounds—not necessarily members of the academic community.

Many popular periodicals are published weekly and almost all of them are published at least monthly.  They tend to be visually appealing with lots of color photographs, graphics, and advertisements.  Almost all popular sources are intended to make a profit, and some of the better known periodicals (Time  or Newsweek, for example) sell millions of copies every week.  Finally, popular sources rarely provide citation information about where the writer got her information.

Generally speaking, academic and non-academic books have characteristics that are similar to academic and non-academic periodicals.  Academic books tend to be written by and for academics, are usually somewhat bland in appearance, tend to be published by companies that are supported by academic institutions, and tend to be only available at academic libraries or specialized bookstores.  Non-academic books tend to be written by journalists or other writers trying to reach a more general audience, they are more eye-catching in appearance, they are published by large and for profit publishing companies, and they are more readily available at public libraries and bookstores.

Scholarly versus Non-Scholarly or Popular Sources

Sources that are Both Scholarly and Non-Scholarly?

While these differences between scholarly and non-scholarly sources might seem straight-forward, many publications are somewhere in between scholarly and non-scholarly.  A journal like College English  is clearly an academic source and a magazine like People  is clearly a popular source.  But categorizing magazines like Ms., Harper’s, or The Atlantic is more difficult since these publications tend to publish articles that are in many ways similar to the articles published in more academic sources.  

Another difficult to categorize source is corporate or “trade” journals.  Most professions and industries have highly specialized publications about that particular business.  For example, Human Resource Executive is targeted to professionals who work in Human Resources departments, Accounting Today is for and about the accounting business, and Advertising Age focuses on the advertising industry.  While most of the writers and editors of trade journals do not have scholarly backgrounds, they tend to be highly focused and knowledgeable about their business. An article about hiring trends in Human Resource Executive will probably have more in common with an academic source than it will with a popular source.

A third “in between” type of research resource is newspapers.  On the one hand, most newspapers would seem to share the characteristics of non-scholarly or popular sources:  they are written for a general audience by writers who are not necessarily experts, they include many photographs and graphics, and so on.  However, a number of publications like The Chronicle of Higher Education are quite different from most newspapers because they are written for a specialized audience, like college and community college teachers and administrators.  Further, newspapers tend to be used by a wide variety of readers and writers-- including scholars-- as a source of basic and reliable information about day-to-day events.

In research writing courses, teachers will often insist students use only or mostly scholarly sources in their research projects because, as is discussed in some detail in the next section in this chapter, scholarly sources tend to be more credible and reliable than non-scholarly sources.   This is not to say that popular sources aren’t credible or reliable; clearly, most of them are, and in many cases, specialized popular sources can be very useful in academic research.  A research project about computer crime may very well include relevant information from a popular source like WIRED or a trade publication written for people who work in the computer industry.  

However, scholarly sources are generally considered more credible and reliable than popular sources.  They tend to publish articles that go into more detail about their subjects, they are written for a more knowledgeable audience, and they are written by experts.

Exercise 1.3

Working alone or collaboratively in small groups, consider the following questions:

•    What sorts of scholarly sources are you and your classmates already familiar with?  What sorts of non-scholarly sources of evidence are you already familiar with that might be useful for your research process?  

•    Think about the kind of topics you are interested in researching and writing about.  Are you aware of any scholarly sources where you are likely to find research on your topic?  What about popular or non-scholarly publications?

•    If you are not yet familiar with specific titles of scholarly or popular sources that might be relevant for your topic, what kind of research would you conduct to find these sources?

The Internet:  The Researcher’s Challenge

Along with the distinction between primary and secondary sources and the distinction between scholarly and non-scholarly publications, you now need to consider a relatively new type of research source as you gather your evidence: the Internet, particularly the World Wide Web.  The Internet started up almost 30 years ago, and elements like electronic mail (“email”) and bulletin board newsgroup discussions have been around for quite some time.  

Widespread use of the Internet really took off in the early 1990s with the development of the World Wide Web and browser software like Mosaic, Netscape, and Internet Explorer.  In fact, the Web has become such a powerful research resource that many beginning research writing students wonder why they should go to the library at all.

Hyperlink:  See the section “What’s ‘a library?’ & ‘What’s The Internet?’” in Chapter 2, “Understanding and Using the Library and the Internet for Research, pages xx-xx.

The Web has become such a powerful medium in part because it has such a far reach—literally, anyone anywhere in the world who is connected to the World Wide Web with the right computer and the right software can access almost any of the hundreds of millions of “pages” and other documents on the Web.  But it also has grown so quickly because it is relatively easy to put documents on to the Web.  In fact, you too might consider exploring some of the options through your school or through a commercial service for joining the World Wide Web community by publishing your research project on the Web.  

Hyperlink:  See the section “The Web-based Research Project” in Chapter 11, “Alternative Ways to Present Your Research.”

Nowadays, the Web has become dominated by corporate and “mainstream” sites that are advertised on television and in traditional magazines and newspapers, which means that it is difficult for an individual’s Web site to compete with the Web sites of The New York Times or amazon.com.  But individuals can still publish their own Web sites, and individually published Web sites can still attract a large and international audience.

Indeed, one of the great strengths of the World Wide Web is that just about anyone can put up “professional looking” Web pages that can reach a potential audience of millions.  However, this strength of the Web is also its weakness, at least as far as being a good place to look for research because anyone can publish what appears to be a “professional” Web site, regardless of his or qualifications.

This fact means the Web is significantly different from more traditional sources of research.  Most scholarly publications are closely scrutinized by editors and other scholars within a particular field.  Further, the articles that appear in even the most non-scholarly of popular sources pass through a variety of different writers and editors before they make it to press.

The problem with many Web pages is that the review process and editors that we assume to be in place with traditional print sources are simply not there. For example, it would be easy for me to fabricate a Web site (complete with charts, graphs, and fake statistics) that argued that students and teachers who used this textbook became more fit, richer, and better-looking.  Such inaccurate claims would never pass the review process of a scholarly journal or a popular magazine--with the possible exception of the sort of tabloid we all see at the grocery store check-out that reports on Elvis sightings.  But on the Web, it is just another page which, if someone finds it “believable,” could be included in someone’s research writing.

The Dihydrogen Monoxide Research Division web site, <http://www.dhmo.org>, certainly looks like an official and reliable web site.  What seems to make it a bit suspect?  What exactly is Dihydrogen Monoxide, anyway?

More seriously, many deceptive and “professional” looking Web pages present very  inaccurate and misleading information and they are not intended to be jokes.  Some of these pages are the work of various hate groups—racists or Holocaust deniers, for example—and some of these sites seem to be the work of con artists.  But when these sites are read uncritically, they can cause serious problems for academic researchers.

Of course, not everything you find on the Web is untrustworthy.  Far from it.  For one thing, the lines between what counts as an Internet source and a more traditional “print” source are beginning to blur.  There are numerous online databases available in many libraries that have complete text versions of articles from academic and popular periodicals, and the articles from these databases are every bit as reliable as the traditional print sources.

Hyperlink:  See the discussion about electronically available periodicals in the section  “Journals, Magazines, and Newspapers”  in Chapter 2, “Understanding and Using the Library and the Internet for Research.”

Additionally, more and more traditional print sources are creating and maintaining Web sites.  Almost all of the most popular news magazines, newspapers, and television networks have Web pages that either reproduce information available in more traditional formats or that publish articles specifically for the Web.  More and more scholarly publications are becoming available on the Web as well, and considering the international reach and low cost of publishing on the Web, it seems inevitable that more (maybe most) academic journals will eventually move from being traditional print journals to ones available only online.

Conversely, not everything you find in traditional print publications—either scholarly or non-scholarly—is always accurate and truthful.  Despite the safeguards that most academic and popular publications follow to ensure they publish truthful and accurate articles, there are all sorts of examples of inaccuracies in print.

More common and therefore perhaps more problematic, small errors and misrepresentations appear in both academic and popular sources, evidence that the process of editorial review is not perfect.  And what “counts” as true or accurate in many fields is a question of some debate and uncertainty, and this is frequently reflected in published articles of all sorts.

Here’s my point:  as I will discuss in the next section of this chapter, the best way to ensure that your evidence is reliable, regardless of where you found that evidence, is to seek out a variety of different types of evidence and to think critically about the quality and credibility of your sources.  This is particularly true with Web-based research.

Exercise 1.4

Working alone or collaboratively in small groups, consider the following questions:

•    Think of a web site that you visit on a regular basis.  What makes this site a useful and credible resource for you?  

•    Are there any Web sites that you have come across that you thought were not believable or credible?  Why did you find this site not believable?

Evaluating the quality and credibility of your research

Finding evidence that answers a question is only the first part of the research process.  You also have to evaluate the quality and credibility of your research.  Inevitably, as we’ve already seen in this chapter, you do this as you consider the origins of your research—primary versus secondary research, scholarly versus popular sources, the Internet, and so forth.  But evaluating the quality and credibility of your research is more subtle and complicated than just determining the source of the evidence.  Consider again the example from the beginning of this chapter about deciding which computer to buy.  One of the things you would have to weigh is the credibility of the information you received from your  friends compared to the information you received from a salesperson at the computer store.  You can probably count on your friends to be trustworthy and honest, but they might not know much about computers.  Conversely, while a salesperson might know a lot about computers, you may be uncertain to what extent you can trust him to give you the best advice.  The salesperson wants to sell you a computer, which means that his motivations might be consciously or unconsciously influencing the information he is providing you.  

Who should you trust?  We have all been in situations like this, and there is no easy way to answer that question.  Chances are, you’ll make your computer decision based on your interpretation of the evidence and based on what you perceive to be the reliability and credibility of your different sources.  If someone else were faced with the same computer decision and the same evidence, they might make a different choice.  That is why there are different kinds of computers on the market and that is why different people can do the same sort of research about “the best” computer and why they can arrive at different conclusions.

Academic research is not much different in the sense that different researchers, considering the same or similar evidence, often arrive at different conclusions.  Academic research rarely provides clear answers in the sense of definitively knowing the “rights” and “wrongs” about some issue.  Not all academics think that computer hacking is wrong (or right), that the solution to commercial over-fishing is strict international control, or that F. Scott Fitzgerald’s novel The Great Gatsby depicts the connection between material goods and the American dream.  Rather, there are debates about these issues, differences of interpretation and opinion that result from different researchers looking at the same evidence.

Furthermore, the debates about differences of opinion on how to interpret evidence are good and healthy because these discussions further our understanding of complex issues.  If we all agreed that something was true, then there would be no point in conducting research and writing about it.  Indeed, if we all agreed about everything and had all of our questions answered as well as we thought possible, there would be no point to education at all!

Ultimately, there is no easy formula for evaluating the credibility and reliability of research.  But there are some basic questions you should ask about your all of your evidence to ensure it is reliable and credible:  

•    Who wrote it?

•    What do you think motivated the writer?

•    Where was it published?

•    When was it written?

Exercise 1.5

Working alone or collaboratively in small groups, consider a variety of different types of research—articles from scholarly and non-scholarly sources, newspaper articles, books, web sites, and other types of evidence.  Using the criteria discussed here, how would you rate the quality and credibility of your research?  Which of your sources seems the most reliable?  Are there any pieces of evidence that, upon closer examination, do not seem credible or reliable?

Evidence Quality and Credibility Checklist

Who wrote or said it?

    •    The writer’s name

    •    Qualifications

    •    Expertise in the field

    •    Previous publications on the topic

    •    Unique experiences of the writer

Why did the source write or say it?

    •    Association with an organization or group

    •    The writer’s stated or implied politics

Where (what source) was it published?

    •    Academic/scholarly source versus non-academic/popular source

    •    Prior knowledge of publication

When was it published or said?

And when it comes to evidence from the ‘net and World Wide Web…

    •    It’s still important to know who wrote it, why you think they wrote it,     where you found it online, and when was it published.

    •    If you don’t know the answers to the who/why/where/when         questions, you should be skeptical of the evidence.

    •    Don’t be fooled by Web sites that “look” real, because…

•    Anybody can publish information on the Web, no matter what that information is.  Unlike most scholarly and many non-scholarly publications, Web writers don’t have to have the work reviewed by editors and publishers to reach an audience.

•    The Internet and the World Wide Web are still good places to find research.  You just have to be a bit more careful with them.