Week 14: one week left; critique; statistics/number logos critical thinking
Last week! Lions and tigers and bears, O my!
SKIM READING ONLY these complex topics linked (more on this in class).
Review/self-check: does your document look like a lemon? Does the reader slide off through your critique into a few, closely related conclusory ideas?
Does your document look more like a pear? Here, the reader moves through your critique into several caveats about conclusions, complexity about conclusions, policy context etc.
Hint: most people will rely on the lemon shape.
Now, onto the hardest analysis piece: evaluating the statistics used to vet the arguments made about data inference. Statistics overview in class this week. I urge you to talk about statistics/logos critical thinking with your science and math professors. To warm up, the ManU Phrasebank includes a "Describe quantities" section. Then, check out the "Reporting results" section, which will help your read your paper's use of statistics or number logos.
You will get better in the future about this critical thinking as you mature as a scientist: Promise! For example, in my field of ecology and environmental science, we are in a quiet riot over frequentist, mutivariate, and Bayesian statistics. This was an assigned reading for me, in one of my classes. Here is another.
For biomedical researchers, you may appreciate this analysis of the limits of p-values in biomedial research.
Please look at your research articles for Wednesday, noting the type of statistics tool/logos of numbers (web exhibit with short definitions) used. Look these up in some way to have a working definition for yourself. Common tools or tests from student papers over the last 15 years include:
- p-values
- confidence intervals
- Student's t test (and corrections)
- analysis of variance (ANOVA); one-tail, two-tail
- power
- sample size
- type of study/limits -- observational study, case note, double-blind
I recommend using the link above to warm up your brain with a short working definition (remember this critical analysis tool from the rain garden memo?) and then go to Wikipedia or even a text book to read about your selected term(s) for more detail.
The pre-reading activity will help you enter into the complexity. Cognitive wedge is also your thinking friend.
I simply want you to know about this area within science articles, even if you do not understand now the statistics. You would not be alone among scientists, if you don't. I don't, in many cases. However, I want you to leave this class with an understanding of this important piece of critical thinking for your field.
One key idea I can wax on about, though, is cautions about the (very limited) definition of significance testing and p-values. For fun, enjoy this comic.
More generally, your critical analysis can comment on findings, your ideas or your close reading the author critique. The ManU phrasebank is really helpful. Here are a few selections that I copy/paste here for you. From the "Being critical" section, see these categories-->
Introducing problems and limitations: theory or argument
Introducing problems and limitations: method or practice
Using evaluative adjectives to comment on research
Introducing general criticism
Introducing the critical stance of particular writers
Practical note on dividing your critique: use separate paragraphs for specific discussion on stats/logos of numbers vetting from your more general critique. For this class, you can pick one limiation to comment on, even though in real life, you would look at more than one weakness. In someways, to focus on one represents a short presentation at a conference. In a seminar setting, you would present more than one weakness. Again, the ManU Phrasebank is so helpful. From "Discussing findings"-->
Advising cautious interpretation of the findings
Another source of uncertainty is …
A note of caution is due here since …
These findings may be somewhat limited by …
These findings cannot be extrapolated to all patients.
These data must be interpreted with caution because …
It could be argued that the positive results were due to …
These results therefore need to be interpreted with caution.
In observational studies, there is a potential for bias from …
It is important to bear in mind the possible bias in these responses.
Although exclusion of X did not …, these results should be interpreted with caution.
However, with a small sample size, caution must be applied, as the findings might not be …
| It is possible that these results | are due to … are limited to … do not represent … have been confounded by … were influenced by the lack of … may underestimate the role of … are biased, given the self-reported nature of … may not be reproducible on a wide scale across … |
Marybeth Shea
Hello to Wednesday. About the statistics thinking we consider, please know that I want to introduce ideas to you that you can consider and refine over the next ten years. One goal is for you to know that even statistics is a place of complex knowledge and procedures that relies on human, expert judgement. Here is a good web article about p-values from Editage. The article is arranged in a list of six points to think critically about this statistical test, the wise application, and how to interpret what p-values mean. This interpretation includes the limits of p-value thinking. Here is my preview definition of the ideas in this piece (cognitive wedge!).
- Using a p-value test can mean that your data set does not fit the related statistical model; this means that the poor values might me you picked the wrong test for your study design. Fix? Consult a statistician in the design phase.
- (I would place this key definition FIRST.) P-values do not measure the probability that the studied hypothesis is true (though thinking this is helpful), or the probability that the data were produced by random chance alone. Instead, p-values really look at the null hypothesis utility. In class, we will talk a bit about scale of vision. At high altitude, we can think of p-values and this testing in this way. However, technically, we have the step of accepting or rejecting the null hypothesis.
- Human judgement matters more than p-values. P-values are part of an exacting critical analysis. Scientific conclusions, by researches and readers, as well as business and/or policy decisions should not be based solely on desired aka low p-values.
- Ethics matter! Proper, robust, and intellectually responsive inference-making requires full reporting and transparency. P-hacking manages to slip through because researchers are not fully honest in their full data set choices and the timing of those choices.
- Statistics help us make meaning. Meaningfulness is not assured by significance testing. A p-value rooted in significance testing does not signal or confirm the importance of a result. Related: a p-value does not measure the size of an effect (For the implications of this huge limitation,see No. 6-->).
- By itself, a p-value does not adequately nor responsibly measure evidence quality; likewise, a p-value can not confirm the intellectual integrity of a study design, supporting model/theory or even the research hypothesis.
Let's talk about power. Many of us look at sample size and conclude the robustness of a finding based in part onn a larger sample size. What is large any way? Depends on research context and even a discipline. You want to ask in the future after you look at sample size this question. How does power work here? Did the researchers even report this important statistical quality? Retuning to Editage, this short piece on statistical power (three minute YouTube explainer by a biostatistian) will help you.
Bottom line: I want you to think about these ideas. Write in the way that you can. I will NOT assess the content for you. Hint: if you plan to use this piece as a writing sample for grad school, either take the stats analysis paragraph out or consult with a mentor in your field.
Train Atlanta? Train Birmingham? Check your ELMS calendar and make your choice. As in a real train, you cannot change trains while in transit.
Marybeth Shea
Happy Friday. I hope you have a change to walk about or bike about (my fave transport) and even drive to see holiday lights. Diwali, Hanukah, and Christmas all share the use of lights to help us (Northern Hemisphere) as the days grow short and nights grow long. Midwinter on the December Solstice (night of Dec 20/sliding into Dec. 21) is the low point of the distribution and then we pivot and climb back out of the darkness toward longer days.
Did you see what I did there? I introduced a data/math assessment of December light+dark volumns. Hah! So did Dan Bridges, software developer. Enjoy his slider visualization and this just-linked Fred Marlton about how sliders work. Both short web exhibits include code so that you can see their wizardry and perhaps adapt for your uses.
--
Come visit today if you wonder about how to do your stats/logos of numbers paragraph and your more general analysis paragraph(s).
- 9-9-50
- 11-11:50
I have a rough short google doc explainer with some phrases concerning ANOVA use. Thanks to JS, for this request. I am happy to make more and share here as well as send to the requesting student. At the end of this explainer, I link to these places that might help some students in this work-->
How to Report Two-Way ANOVA Results (With Examples)
How to Report t-Test Results (With Examples)
How to Report Chi-Square Results (With Examples)
How to Report Pearson’s Correlation (With Examples)
How to Report Regression Results (With Examples)
The source of these above links is Statology, a consulting group that shares knowledge as way to build cllient relationships with people needing stats support. Note: the language here reflects how a researcher can write about their use of tests as they report results, draw inferences, and defend their conclusions in an article. You can use this writing as mentor sentences and phrases for how you can comment using their writing. Use first person voice to make clear when you are speaking, even if you invoke your researchers. Then, you can shift to third person.
OWL is also helpful for you as you read about statistics and write about this important aspect of science research reporting.
- WRITING WITH STATISTICS
- Quick Tips On Writing with Statistics
- Descriptive Statistics
- Writing with Descriptive Statistics
- Basic Inferential Statistics: Theory and Application
- Writing with Inferential Statistics
- Statistics and Visuals
- Key Terms
More on p-values at Khan Academy. Here is one- (7-minutes video plus time-divided guide)->
DO NOT FORGET THE ASSIGNMENTS TONIGHT FOR BOTH TRAIN Atlanta and TRAIN Boulder! You have ER Writing Task links in your ELMS calendar and in a class email to all of you, within ELMS.
Post a Comment
Reader Comments