Critically Reviewing A Research Paper

When considering a research idea, we are bound to rely on previous findings on the topic. Work done in the field constructs the foundation for our research and determines its course and value. Inaccurate findings may lead to imprecise applications and end in further fallacies in your own new scientific knowledge that you construct.  In order to set a solid basis for research on any topic and to prevent multiplication of misinformation, it is crucial to to critically evaluate existing scientific evidence. It is important to know which information can be regarded as plausible.

So what’s the criteria to determine whether a result can be trusted? As it is taught in the first classes in psychology, errors may emerge from any phase of the research process. Therefore, it all boils down to how the research has been conducted and the results presented.

Meltzoff (2007) emphasizes the key issues that can produce flawed results and interpretations and should therefore be carefully considered when reading articles. Here is a reminder on what to bear in mind when reading a research article:

Research question
The research must be clear in informing the reader of its aims. Terms should be clearly defined, even more so if they’re new or used in specific non-spread ways. You as a reader should pay particular attention should to errors in logic, especially those regarding causation, relationship or association.

Sample
To provide trustworthy conclusions, a sample needs to be representative and adequate. Representativeness depends on the method of selection as well as the assignment.  For example, random assignment has its advantages in front of systematic assignment in establishing group equivalence. The sample can be biased when researchers used volunteers or selective attrition. The adequate sample size can be determined by employing power analysis.

Control of confounding variables
Extraneous variation can influence research findings, therefore methods to control  relevant confounding variables should be applied.

Research designs
The research design should be suitable to answer the research question. Readers should distinguish true experimental designs with random assignment from pre-experimental research designs.

Criteria and criteria measures
The criteria measures must demonstrate reliability and validity for both, the independent and dependent variable.

Data analysis
Appropriate statistical tests should be applied for the type of data obtained, and assumptions for their use met. Post hoc tests should be applied when multiple comparisons are performed. Tables and figures should be clearly labelled. Ideally, effect sizes shou

ld be included throughout giving a clear indication of the variables’ impact.

Discussion and conclusions
Does the study allow generalization? Also, limitations of the study should be mentioned. The discussion and conclusions should be consistent with the study’s results. It’s a common mistake to emphasizing the results that are in accordance with the researcher’s expectations while not focusing on the ones that are not. Do the authors of the article you hold in hand do the same?

Ethics
Last but not least, ere the ethical standards met? For more information, refer to the APA’s Ethical Principles of Psychologists and Code of Conduct (2010).

References
American Psychological Association (2010, June 1). American Psychological Association Ethical Principles of Psychologists and Code of Conduct. Retrieved July 28, 2011 from http://www.apa.org/ethics/code/index.aspx

Meltzoff, J. (2007). Critical Thinking About Research. Washington, DC: American Psychological Association.

 

Edited by: Maris Vainre

Posted in How-to, Literature research and tagged literature review, literature search, review process, scientific writing style, tips on by Zorana Zupan. 6 Comments

Writing a critical review of a journal article can help to improve your research skills. By assessing the work of others, you develop skills as a critical reader and become familiar with the types of evaluation criteria that will be applied to research in your field and thus your own research.

You are expected to read the article carefully, analyse it, and evaluate the quality and originality of the research, as well as its relevance and presentation. Its strengths and weaknesses are assessed, followed by its overall value. Do not be confused by the term critique: it does not mean that you only look at the negative aspects of what the researcher has done. You should address both the positive and negative aspects.

If your lecturer has given you specific advice on how to write a critical review, follow that advice. If not, the following steps may help you. These steps are based on a detailed description of how to analyse and evaluate a research article provided by Wood (2003) in her lab guide.

This guide is divided into two parts. The first part, "Researching the Critique," outlines the steps involved in selecting and evaluating a research article. The second part, "Writing your Critique," discusses two possible ways to structure your critique paper.

A. Researching the Critique

The questions listed under many of the subheadings in this section may provide you with a good place to begin understanding what you are looking for and what form your critique might take.

1. Select a Topic

If your lecturer does not assign a topic or a particular article for you to review, and you must choose a topic yourself, try using a review article from your field. Review articles summarize and evaluate current studies (research articles) on a particular topic. Select a review article on a topic that interests you and that is written clearly so you can understand it.

2. Select a Research Article

Use the review article to select a research article. This can be very useful in writing your critique. The review article will provide background information for your analysis, as well as establishing that the research paper you are critiquing is significant: if the paper was not so highly regarded, it would not have been selected to be reviewed.

When choosing a research article, examine the Materials & Methods section closely and make sure you have a good grasp of the techniques and methods used. If you don't, you may have difficulty evaluating them.

3. Analyse the Text

Read the article(s) carefully. As you read the article(s) use the following questions to help you understand how and why the research was carried out.

  • What is the author's central purpose? Look at INTRODUCTION.
  • What methods were used to accomplish this purpose (systematic recording of observations, analysis and evaluation of published research, assessment of theory)? Look at METHODS.
    What were the techniques used? and how was each technique performed?
    What kind of data can be obtained using each technique?
    How are such data interpreted?
    What kind of information is produced by using the technique?
  • What objective evidence was obtained from the author's efforts (observations, measurements etc.)? What were the results of the study? Look at RESULTS.
    How was each technique used to obtain each result?
    What statistical tests were used to evaluate the significance of the conclusions based on numeric or graphic data?
    How did each result contribute to answering the question or testing the hypothesis raised in the introduction?
  • How were the results interpreted? How were they related to the original problem (author's view of evidence rather than objective findings)? Look at DISCUSSION.
    Were the author(s) able to answer the question (test the hypothesis) raised?
    Did the research provide new factual information, a new understanding of a phenomenon in the field, a new research technique?
    How was the significance of the work described?
    Did the reported observations/interpretations support or refute observations or interpretations made by other researchers?

(Adapted with permission of Professor Susan Lollis, Family Relations and Applied Nutrition, University of Guelph. Source of questions in each section Wood, 2003)

4. Establish the Research Context

Once you are reasonably familiar with the article, it is important to gain an understanding of the research context, both societal and intellectual. To establish the research context, questions such as the following should be addressed:

  • Who conducted the research? What were/are their interests?
  • When and where was the research conducted?
  • Why did they do this research?
  • Was this research pertinent only within the authors' geographic locale, or did it have broader (even global) relevance?
  • Were many other laboratories pursuing related research when the reported work was done? If so, why?
  • For experimental research, what funding sources met the costs of the research?
  • Was the selection of the research topic influenced by the source of research funding?
  • On what prior observations was the research based? What was and was not known at the time?
  • How important was the research question posed by the researcher?

For more detailed information on how to answer these questions, see Labs 4 and 5 (Wood, 2003).

5. Evaluate the Text

After you have read the article and answered the questions in the previous section, you should have a good understanding of the research undertaken. You can now begin to evaluate the author's research. Making judgements about someone else's work is often the most difficult part of writing the review. Many students feel that, because they are new to a discipline, they do not have enough knowledge to make judgements of other people's work.

The following checklist may assist you:

INTRODUCTION

  • Read the statement of purpose at the end of the introduction. What was the objective of the study?
  • Consider the title. Does it precisely state the subject of the paper?
  • Read the statement of purpose in the abstract. Does it match the one in the introduction?
  • Check the sequence of statements in the introduction. Does all the information lead coherently to the purpose of the study?

METHODS

  • Review all methods in relation to the objective(s) of the study. Are the methods valid for studying the problem?
  • Check the methods for essential information. Could the study be duplicated from the methods and information given?
  • Check the methods for flaws. Is the sample selection adequate? Is the experimental design sound?
  • Check the sequence of statements in the methods. Does all the information belong there? Is the sequence of methods clear and pertinent?

RESULTS

  • Examine carefully the data as presented in the tables and diagrams. Does the title or legend accurately describe the content? Are column headings and labels accurate? Are the data organized for ready comparison and interpretation? (A table should be self-explanatory, with a title that accurately and concisely describes content and column headings that accurately describe information in the cells.)
  • Review the results as presented in the text while referring to the data in the tables and diagrams. Does the text complement, and not simple repeat, data? Are there discrepancies between the results in the text and those in the tables?
  • Check all calculations and presentation of data.
  • Review the results in light of the stated objectives. Does the study reveal what the researcher intended?

DISCUSSION

  • Check the interpretation against the results. Does the discussion merely repeat the results? Does the interpretation arise logically from the data or is it too far-fetched? Have the faults/flaws/shortcomings of the research been addressed?
  • Is the interpretation supported by other research cited in the study?
  • Does the study consider key studies in the field?
  • Are there other research possibilities/directions suggested?

OVERVIEW

  • Reread the abstract. Does it accurately summarize the article?
  • Check the structure of the article (first headings and then paragraphing). Is all the material organized under the appropriate headings? Are sections divided logically into subsections or paragraphs?
  • Are stylistic concerns, logic, clarity and economy of expression addressed?

(adapted from Kuyper, 1991)

6. Establish the Significance of the Research

Finally, it is important to establish whether the research has been successful – has it led to new questions being asked, new ways of using existing knowledge? Are other researchers citing this paper?

The following questions should be answered:

  • How did other researchers view the significance of the research reported by your authors?
  • Did the research reported in your article result in the formulation of new questions or hypotheses (by the authors, by other researchers)?
  • Have other researchers subsequently supported or refuted the observations/interpretations of these authors?
  • Did the research make a significant contribution to human knowledge?
  • Did the research produce any practical applications?
  • What are the social, political, technological, medical implications of this research?
  • How do you evaluate the significance of the research?

To answer these questions look at review articles to find out how reviewers see this piece of research. Look at research articles to see how other people have used this work; what range of journals have cited this article? For more detailed information on how to answer these questions, see Lab. 8 (Wood, 2003).

B. Writing your Critique

Two possible approaches

You have completed your analysis and evaluation of the journal article. How do you then put all this information together? If your instructor has not provided a format for your critique, there are two possible ways you might present it.

Approach (A)

If your instructor is concerned that that the article be clearly situated within the social and intellectual research context, then you might present it in the following way:

Introduction

In the introduction, cite the journal article in full and then provide the background to this piece of research, establishing its place within the field. Use the answers to the questions in Establish the Research Context to develop this section.

Body

Follow the structure of the journal article. Evaluate each section of the article — Introduction, Methods, Results, Discussion — highlighting the strengths and weaknesses of each section. Use the answers to the questions in Evaluate the Text to develop this section.

Conclusion

In this section, sum up the strengths and weaknesses of the research as a whole. Establish its practical and theoretical significance. Use the answers to questions Establish the Significance of the Research to develop this section.

Approach (B)

Another common way to structure a journal article critique is the following:

Introduction

In the introduction, cite the journal article in full and provide a summary of the journal article. Use the answers to the questions in the section Analyze the Text to develop the summary.

Body

Follow the structure of the journal article. Evaluate each section of the article – Introduction, Methods, Results, Discussion – highlighting the strengths and weaknesses of each section. Use the answers to the questions in Evaluate the Text to develop this section.

Conclusion

In this section, sum up the strengths and weaknesses of the research as a whole. Establish its practical and theoretical significance. Use the answers to questions Establish the Significance of the Research to develop this section.

References

Kuyper, B.J. (1991). Bringing up scientists in the art of critiquing research. Bioscience 41(4), 248-250.

Wood, J.M. (2003).Research Lab Guide. MICR*3260 Microbial Adaptation and Development Web Site. Retrieved July 31, 2006, from http://www.uoguelph.ca/mcb/teaching/micr3260/research_lab/guide.shtml

Categories: 1

0 Replies to “Critically Reviewing A Research Paper”

Leave a comment

L'indirizzo email non verrà pubblicato. I campi obbligatori sono contrassegnati *