Assignment: Evaluating Significance of Findings

 
Part of your task as a scholar-practitioner is to act as a critical consumer of research and ask informed questions of published material. Sometimes, claims are made that do not match the results of the analysis. Unfortunately, this is why statistics is sometimes unfairly associated with telling lies. These misalignments might not be solely attributable to statistical nonsense, but also “user error.” One of the greatest areas of user error is within the practice of hypothesis testing and interpreting statistical significance. As you continue to consume research, be sure and read everything with a critical eye and call out statements that do not match the results.

For this Assignment, you will examine statistical significance and meaningfulness based on sample statements.

To prepare for this Assignment:

• Review the Week 5 Scenarios found in this week’s Learning Resources and select two of the four scenarios for this Assignment.
• For additional support, review the Skill Builder: Evaluating P Values and the Skill Builder: Statistical Power, which you can find by navigating back to your Blackboard Course Home Page. From there, locate the Skill Builder link in the left navigation pane.

For this Assignment:

Critically evaluate the two scenarios you selected based upon the following points:

• Critically evaluate the sample size.
• Critically evaluate the statements for meaningfulness.
• Critically evaluate the statements for statistical significance.
• Based on your evaluation, provide an explanation of the implications for social change.

Use proper APA format and citations, and referencing.

 ***also you must receive a 90 percent or better on this assignment***
 

 

Required Readings

Frankfort-Nachmias, C., Leon-Guerrero, A., & Davis, G. (2020). Social statistics for a diverse society (9th ed.). Thousand Oaks, CA: Sage Publications.
Chapter 8, “Testing Hypothesis: Assumptions of Statistical Hypothesis Testing” (pp. 241-242)
Wagner, III, W. E. (2020). Using IBM® SPSS® statistics for research methods and social science statistics (7th ed.). Thousand Oaks, CA: Sage Publications.
Chapter 6, “Testing Hypotheses Using Means and Cross-Tabulation”
Warner, R. M. (2012). Applied statistics from bivariate through multivariate techniques (2nd ed.). Thousand Oaks, CA: Sage Publications.
Applied Statistics From Bivariate Through Multivariate Techniques, 2nd Edition by Warner, R.M. Copyright 2012 by Sage College. Reprinted by permission of Sage College via the Copyright Clearance Center.
Chapter 3, “Statistical Significance Testing” (pp. 81–124)
Applied Statistics From Bivariate Through Multivariate Techniques, 2nd Edition by Warner, R.M. Copyright 2012 by Sage College. Reprinted by permission of Sage College via the Copyright Clearance Center. 
Magnusson, K. (n.d.). Welcome to Kristoffer Magnusson’s blog about R, Statistics, Psychology, Open Science, Data Visualization [blog]. Retrieved from http://rpsychologist.com/index.html
As you review this web blog, select [Updated] Statistical Power and Significance Testing Visualization link, once you select the link, follow the instructions to view the interactive for statistical power. This interactive website will help you to visualize and understand statistical power and significance testing.
Note: This is Kristoffer Magnusson’s personal blog and his views may not necessarily reflect the views of Walden University faculty.
American Statistical Association (2016). American Statistical Association Releases Statement on Statistical Significance and P-Values. Retrieved from http://www.amstat.org/newsroom/pressreleases/P-ValueStatement
As you review this press release, consider the misconceptions and the misuse of p-values in quantitative research.
Document: Week 5 Scenarios (PDF)
Use these scenarios to complete this week’s Assignment.

© 2016 Laureate Education, Inc. Page 1 of 2

Week 5

Scenarios

1. The p-value was slightly above conventional threshold, but was described as
“rapidly approaching significance” (i.e., p =.06).

An independent samples t test was used to determine whether student satisfaction
levels in a quantitative reasoning course differed between the traditional classroom
and on-line environments. The samples consisted of students in four face-to-face
classes at a traditional state university (n = 65) and four online classes offered at
the same university (n = 69). Students reported their level of satisfaction on a five-
point scale, with higher values indicating higher levels of satisfaction. Since the
study was exploratory in nature, levels of significance were relaxed to the .10 level.
The test was significant t(132) = 1.8, p = .074, wherein students in the face-to-face
class reported lower levels of satisfaction (M = 3.39, SD = 1.8) than did those in the
online sections (M = 3.89, SD = 1.4). We therefore conclude that on average,
students in online quantitative reasoning classes have higher levels of satisfaction.
The results of this study are significant because they provide educators with
evidence of what medium works better in producing quantitatively knowledgeable
practitioners.

2. A results report that does not find any effect and also has small sample size
(possibly no effect detected due to lack of power).

A one-way analysis of variance was used to test whether a relationship exists
between educational attainment and race. The dependent variable of education
was measured as number of years of education completed. The race factor had
three attributes of European American (n = 36), African American (n = 23) and
Hispanic (n = 18). Descriptive statistics indicate that on average, European
Americans have higher levels of education (M = 16.4, SD = 4.6), with African
Americans slightly trailing (M = 15.5, SD = 6.8) and Hispanics having on average
lower levels of educational attainment (M = 13.3, SD = 6.1). The ANOVA was not
significant F (2,74) = 1.789, p = .175, indicating there are no differences in
educational attainment across these three races in the population. The results of
this study are significant because they shed light on the current social conversation
about inequality.

3. Statistical significance is found in a study, but the effect in reality is very small (i.e.,
there was a very minor difference in attitude between men and women). Were the
results meaningful?

An independent samples t test was conducted to determine whether differences
exist between men and women on cultural competency scores. The samples
consisted of 663 women and 650 men taken from a convenience sample of public,
private, and non-profit organizations. Each participant was administered an
instrument that measured his or her current levels of cultural competency. The

© 2016 Laureate Education, Inc. Page 2 of 2

cultural competency score ranges from 0 to 10, with higher scores indicating higher
levels of cultural competency. The descriptive statistics indicate women have
higher levels of cultural competency (M = 9.2, SD = 3.2) than men (M = 8.9, SD =
2.1). The results were significant t (1311) = 2.0, p <.05, indicating that women are more culturally competent than are men. These results tell us that gender-specific interventions targeted toward men may assist in bolstering cultural competency.

4. A study has results that seem fine, but there is no clear association to social
change. What is missing?

A correlation test was conducted to determine whether a relationship exists
between level of income and job satisfaction. The sample consisted of 432
employees equally represented across public, private, and non-profit sectors. The
results of the test demonstrate a strong positive correlation between the two
variables, r =.87, p < .01, showing that as level of income increases, job satisfaction increases as well.

Running head: EVALUATING FINDINGS

EVALUATING FINDINGS 2

Scenario 1; Qualitative Analysis

Jamisha

Walden University

After collecting my data which is quantitative, a statistical equation should be utilized to give an estimate which is rough of the example size that is wanted. There are consistently errors in inspecting that mirror the fluctuations that emerge among measurements and the population that is being utilized in the example is certainly not an exact portrayal of the whole population that is being tended to. On the off chance that the analysts were to expand the example size, the danger of blunder can be limited or near being dispensed with if the whole population was to be utilized.

In the principal situation, a test was finished to decide whether an understudy’s fulfillment levels varied between the customary study hall and online meetings. The examples comprised of understudies that were in four face-to-face classes at the state college and four online classes at a similar college. For the face-to-face classes, there were 65 understudies that were studied, and for the on the web, there were 69 understudies. The understudies were approached to rate on a five-point scale with the higher rating having a more significant level of fulfillment.

Statistical essentialness manages the basic estimation of a measurement. What’s more, in a specific way of thinking, making an assurance of whether the invalid theory is dismissed or you neglect to dismiss the invalid speculation. Meaningfulness is taking that statistic and determining its applicability out in the real world. In this scenario the statistical significance would be to assess the significance of the test between the classroom which would be the control group and the online students which would be the experimental group. In the real world this sampling could be linked to satisfaction overall. If the level of satisfaction is increased then the level of being successful in college would be greater.

The assertion, “The test was critical t (132) = 1.8, p = .074, wherein understudies in the face-to-face class announced lower levels of fulfillment (M = 3.39, SD = 1.8) than did those in the online areas (M = 3.89, SD = 1.4)”, sounds very confusing and misleading (Wagner, 2016). It would be better to break each of the two groups into smaller groups. It is also recommended that the threshold of 0.05 be used. The implications for social change would be to avoid the errors that have been identified when doing this type of research in the future. The response that each of the students give will help with what resources or styles of teaching are more effective with the students to be the most successful that they can be. Once the student is successful their overall demeanor can be more positive on society as a whole.

In scenario two, the independent variable is race and the dependent variable is education. In this scenario it determines that there is not a difference in the education that is received across each of the races. The sample size is 36 European Americans, 23 African Americans, and 18 Hispanics. .175 is the p-value which would demonstrate that the specialist ought to neglect to dismiss the invalid theory and express that there is no statistical noteworthiness where the p-esteem is more prominent than .05.

The meaningfulness in this scenario would be that the current social conversation is about inequality. The result was that there were no distinctions in instructive fulfillment across the three races. The ANOVA shows that there are no distinctions in instructive achievement across the races while the numbers show contrastingly for each race. The social change would be that resources need to be put in place to determine if there are programs that are needed for the races that are not having the higher number such as English-speaking programs. Many individuals that are of Hispanic decent cannot speak English. They may also need more individualized help with filling out paperwork for college.

References

Frankfort-Nachmias, C., & Leon-Guerrero, A. (2018). Social statistics for a diverse society

(8th ed.). Thousand Oaks, CA: Sage Publications.

Wagner, W. E. (2016). Using IBM® SPSS® statistics for research methods and social science

statistics (6th ed.). Thousand Oaks, CA: Sage Publications.