====== 810xs | Week 4 | Evaluating Research Reports ======

Your readings in the Beck & Martin book have been about research in general and about conducting research, but we're now reading Pyrczak on how to evaluate research.

From your experience with the application exercise, please share what you think are the hardest parts of the evaluation process? Will these always be the same? IS there anything that could make this easier?
----
**
From your experience with the application exercise, please share what you think are the hardest parts of the evaluation process?**

Initially, the hardest part of the evaluation process is general unfamiliarity with the research domain and its theory base, the various research methodologies and methods used, and the formal elements (i.e., measures, experimental procedures descriptions, analysis and results sections) of journal research articles. For a novice, each element provides a degree of challenge commensurate with previous exposure to it. For example, lacking a foundation in the various statistical methods applied means a high level of acceptance on faith (rather than knowledge) that the method was applied correctly, was applied to an appropriate problem, and that researcher claims of significance are reasonable.

With exposure and further study, the evaluation process will become easier. I felt more confidence about being able to say something intelligent and relevant after reading the research article I used for the application exercise in Pyrczak. I also constructed a rough-draft visual map DIAGRAM_ResearchEvaluation.jpeg (Links to an external site.) to help me think about these topics holistically and for future reference.

**Will these always be the same?**

Short response - No. Because knowledge is acquired unevenly there will be some areas that seem to be intuited quicker than others. Doing the sampling exercise in during the class weekend and then discussing our various outcomes and observations made judging those elements of the research clearer than say the validity of the researcher's claims of statistical significance for that same sample.

**IS there anything could make this easier?**

Expertise gained through experience is the most obvious response. But evaluation tools would be my preferred approach to making the process easier. Whether it is as basic as a list of questions like those provided by Pyrczak or the reading of a textbook on Practical Research Methods for Librarians and Information Professionals, tools that focus attention on problem areas (e.g., what are the problem areas in sampling methods, etc.) are efficient ways of simplifying the evaluation process. I would also keep in mind that that beyond the known unknowns there are likely to be unknown unknowns that are “invisible” to the research evaluation process. This level of analysis would include thinking about the theory that underlies the research.  But for most practical evaluations, this won’t be necessary--especially for a novice.

Here is the APA reference of the article I used for the application exercise in Pyrczak:

Mahmood, K. Do people overestimate their information literacy skills? A systematic review of empirical evidence on the Dunning-Kruger effect. Communications in Information Literacy, 10(2), 198-213. 

Link to resource: https://emporiastate.idm.oclc.org/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=llf&AN=120468655&site=ehost-live (Links to an external site.)

Cut and paste into browser.