INTRODUCTION

The Program and the Problem

The Produce Prescription (Rx) program serves patients who utilize a local free clinic, have incomes below the federal poverty line, are uninsured, and have a diet-related health condition. This program is run by a team of Registered Dietitians (RDs) who specialize in providing nutrition-related services [1]. Participants receive nutrition education, cooking demos, recipes, and a bag of local produce (their nutrition prescription). Their nutrition prescription provides enough produce for participants to eat two servings each day for fourteen days. The hope is to improve disease maintenance through nutrition education and the promotion of plant-based diets. However, the program is experiencing irregular levels of attendance. Even though the program provides free services, it must pay for the produce and packaging materials. Irregular attendance patterns threaten the fidelity and sustainability of the program. It could also impact the health outcomes seen in the participants. The fidelity of the program and health outcomes of the participants are impacted because it may appear that the program is ineffective, when the intervention was not implemented (they did not get their nutrition prescription). The sustainability of the program is impacted because along with improved disease maintenance, the program seeks to lower the amount of medication needed to manage chronic health conditions, which the clinic also provides for free. Reducing the participants medications saves the clinic money and increases the sustainability of both the clinic and the Produce Rx program.

The Solution to the Program

The Produce Rx program seeks to create a Thurstone scale. Thurstone scales are used to measure attitudes towards a particular focus. The focus of the Hamilton Healthy Eating Attitude Thurstone (HEAT) scale is to measure favorable and unfavorable attitudes towards fruit and vegetable consumption. This will be used to screen future candidates of the program and assist in selecting participants who have higher likelihoods of adherence. If the Produce Rx program can select participants with higher likelihoods of adherence and attendance, this will improve the fidelity, sustainability, and possible health impacts of the program.

Thurstone Scales

Attitudes are expressions of an individual’s views on physical, cognitive, and affective aspects of their lives. Measuring these constructs often require the use of a scale that determines favorable and unfavorable attitudes at different continuums. Thurstone scales were the first of their kind and contain four vital phases after the main construct is selected: item creation, an item analysis, statistical rigor, and pilot testing. Item creation should result in an ample number of statements that relate to the construct. Judges or experts in the construct being measured should perform the item analysis, which includes ranking each item on a range of favorable to unfavorable. Statistical rigor should reduce the final scale to eleven to twenty-two items. Pilot testing helps to understand how well the scale works on the target population and if changes need to be made. Throughout this process the researcher must remain alert to context, wording, the construct being measured, and additional factors. Scales that measure attitudes are used across multiple disciplines [2]. Thurstone scales have recently been created to measure buying habits and attitudes of food company stakeholders and even to evaluate the dietary habits seen in those at-risk for head and neck cancer [3-4]. This shows the diversity of Thurstone scales. This research project will highlight the creation and validation of the H-HEAT scale in relevance to the Produce Rx program

METHODS

Item Creation

The initial list of items was generated by reviewing comments made my Produce Rx participants. Individuals who work within and outside the healthcare field were also consulted. Eighty items were created due to these actions. Additionally, national questionnaires and smaller studies assisted in item generation [5-9]. None of the studies had identical aims to this research project. However, they were similar enough to make comparisons and gather ideas. The scale now had one-hundred and twenty-five items. The next step was to use the Health Belief model (HBM) to classify each item [10]. This is used to predict health-related behaviors in individuals and is commonly used in health and social sciences. Each of the one-hundred and twenty-five items were characterized as perceived susceptibility (n = 7), perceived severity (n = 15), perceived benefits (n = 25), perceived barriers (n = 23), cues to action (n = 21), and self-efficacy (n = 34).

Experts in creating Thurstone scales (n = 2) were contacted and members of the research team were consulted concerning the next steps. The main suggestions were to lower the reading level, make each item a strong indicator of positive or negative attitudes, and reduce the number of items. After reviewing the current scale and referencing literature, the Flesch-Kincaid Reading Ease Level was increased (62.2 to 84.7), the grade level was lowered (7.9 to 3.0), and the scale was reduced from one-hundred and twenty-five to eighty items.

Once the Clemson University Office of Research Compliance granted approval, the researchers began to create the item-analysis and questionnaire using Qualtrics. It was decided that conducting the item-analysis should be done online because of the status of the Coronavirus Disease 2019 (COVID-19) pandemic in the United States [11-12]. Additionally, this enabled researchers to reach individuals across a wider geographical location. Since the item-analysis was online, extra notes were added as guidance throughout the questionnaire. Respondents were asked to rate each item on a scale of 1.0 (very unfavorable) to 11.0 (very favorable). The item-analysis could be completed using a computer or smart-device. A total of fifty-six individuals were contacted via email, text message, and Facebook Messenger to complete the questionnaire. Information regarding these respondents can be found in Table 1 and Table 2.

Pilot Testing and Cognitive Interviews

The H-HEAT scale was pilot tested on two groups. The first group consisted of junior and senior Human Nutrition students at Clemson University. Based on their degree requirements, it was determined that they would be expected to have favorable attitudes towards fruit and vegetable consumption [13].

The other group were patients of a local clinic. They were nearly identical to the patients of the Clemson Free Clinic (the target population); low-income, uninsured, most had a diet-related health condition, and they utilized a community-based free healthcare facility [14]. It was hypothesized that the students would have higher scores than the patients of the local free clinic.

After reviewing information about the COVID-19 pandemic and reviewing research concerning over the phone interviews, it was determined that conducting the pilot tests and cognitive interviews over the phone would be the most appropriate option [15]. The script was developed in accordance with requirements provided by Clemson University’s Office of Research Compliance [16]. Messages were sent to the directors of the Human Nutrition program at Clemson University and the local clinic (via email and phone). The cognitive interviews and pilot testing took place from July to September in 2020.

RESULTS

There was a 69% response rate for the initial item-analysis. Items from each Median value (1.0-11.0) were selected based on how small their Interquartile Range (IQR) was. In the case of a “tie” both items were included. This process reduced the eighty-item scale to twenty, which was then pilot tested on patients of a local clinic and Human Nutrition students. A 44% response rate was obtained from individuals at the local free clinic. Their demographic information and scores can be seen in Table 3. Additionally, six Human Nutrition students participated. Their demographics and scores can be seen in Table 4. The twenty-item scale took an average of eight and a half minutes to complete (including probing questions). Items were removed based on the results of the cognitive interviews and conducting an item-total statistic. The final twelve-item scale can be seen in Table 5. It had a Cronbach’s alpha of .07. The finalized scale was used to recalculate the highest and lowest scores from the students and patients of the local clinic. The scores of individuals with favorable scores increased and vice versa with those who had less favorable scores. The results can be seen in Table 6

DISCUSSION

Item-Analysis Response Rates

Participants were given fourteen days to complete and submit their responses. One individual completed the item-analysis after it closed, which resulted in the removal of their data. Another individual reached out and thought the double negative questions (a positive and negative attitude and a choice of 1.0-11.0) were confusing. However, that issue was resolved. Qualtrics estimated that it would take 20 minutes to complete the analysis. However, the results revealed that it took participants an average of 34 minutes to complete.

Pilot Test and Cognitive Interview Reflections

Grunts, sighs, and other verbal expressions typically associated with frustration were heard during the interviews. These individual were not asked follow-up questions in fear that they would become too distressed and withdrawal from the study. A few individuals asked the facilitator to explain the purpose of the study more in-depth due to misinterpretations of what was being asked of them. One individual thought there was an actual test involved (due to it being called a pilot test). Perhaps some of the terminology in the opening statement needed to be simplified to the reading level of the scale. Additionally, longer items were asked to be repeated more often. Perhaps they are too long for a phone-administered scale.

The most concerning comment and expression received was laughter. Research shows that how an interviewer responds to laughter invitations or provides pseudo laughter can impact the study [17]. Laughter can reduce how serious a respondent takes the study and increases the likelihood of respondent bias. Some research organizations go as far to prohibit laughter (from the administrator) done over the phone [18]. Pseudo laughter occurred with the patients of the local clinic but not the Human Nutrition students.

Calculating the H-HEAT Scale Score

To use the scale, add the sum the scores of the items that the respondent agrees with and divide by the number of items they agreed with.

IMPLICATIONS FOR RESEARCH AND PRACTICE

Threats to Internal and External Validity

Threats to internal and external validity include the history threat, selection bias, the mortality threat, and the instrumentation threat [19]. All the participants participated in the study during the COVID-19 pandemic. However, they took it at different times and dates. Additionally, some of the students participated before and after the start of the 2020 fall semester. These outside events can be linked to the history threat and may have impacted the data. Selection bias may have occurred when the health professionals were selected to conduct the item-analysis. These individuals were not randomly selected. They were limited to who the lead researcher knew on a professional basis. Additionally, an invitation was not sent out to all healthcare professionals. The mortality threat occurred while the pilot tests and cognitive interviews were being administered at the local clinic. A total of three individuals participated but decided to withdrawal after the confidentiality statement was read. All these factors could reduce the validity of the study. The instrumentation threat was avoided while the pilot test and cognitive interviews were administered because the same researcher and scale were used.

Future Uses, Advancing the Field, and COVID-19

Although research involving the creation and use of Thurstone scales exist, there is limited data on such scales being used on low-income, uninsured, and multimorbidity populations. There is also limited information regarding the effectiveness of a food prescription program like the one highlighted by this research project. In many aspects, the Produce Rx program and the H-HEAT scale can serve as a pilot for other programs seeking to address similar topics. This advances the field of nutrition and dietetics by adding to the growing body of research and allowing similar programs to avoid pitfalls in their implementations. This is especially important due to the impacts of COVID-19. A study showed that individuals reported having increased ingestion of sweets, processed meats, and other undesirable food choices. It also showed decreases in dairy intake, whole grain products, and other desirable foods. However, there were reports of increased water intake. The study related this to varying levels of food security during COVID-19 [20]. However, adding the elements of attitudes towards desirable and undesirable food choices, which could be done with the H-HEAT scale, would add a rich source of data to the field.

CONCLUSION

The H-HEAT scale measures favorable and unfavorable attitudes towards fruit and vegetable consumption. It was designed to be used as a screening tool for admission into the Produce Rx program, whose fidelity, health impact, and sustainability were threatened by irregular attendance patterns. Selecting participants with favorable H-HEAT scale scores would ideally alleviate this issue. Additionally, the Produce Rx program and the H-HEAT scale serve as pilots for similar programs to allow them to avoid pitfalls in their implementation and add to the growing body of research.

ETHICAL STATEMENT

Permission to conduct this research project was obtained from the Clemson University Office of Research Compliance in 2020. All participants consented to participate and gave verbal or electronic confirmation. Protocol for obtaining consent, provided by Clemson University, were followed.

ACKNOWLEDGMENTS

Derek Nazareth, PhD, MBA: Participating Investigator

Edward Ip, PhD: Participating Investigator

Beth Kunkel, PhD, RD, FADA : Participating Investigator

Sarah Griffin, PhD, MPH: Participating Investigators

This material is based upon work supported by the NIFA/USDA, under project number NC-1196.

CONFLICTS OF INTEREST

All member of the research team or employed by Clemson University.

REFERENCES

1. eatright.org [Internet]: Academy of Nutrition and Dietetics; c2021 [cited 2021 Jan 27]. Available from: https://www.eatrightpro.org/about-us/what-is-an-rdn-and-dtr/what-is-a-registered-dietitian-nutritionist
2. Rosenberg B, Silva T. The sage encyclopedia of educational research, measurement, and evaluation attitude scaling. SAGE Publications, Inc 2018 Feb 26: 140-144.
3. Banerjee S. Role of food companies to supply nutritious foods as per buyers changing lifestyles, buying habits and the recent trends. J INNOV 2020; 9(3): 1062-1067.
4. Vito R, Lee Y, Parpinel M, Serranio D, Olshan A, Zevallos JP, Levi F, Zhang ZF, Morgenstern H, Garavello W, Kelsey K, McClean M, Schantz S, Yu GP, Boffetta P, Chuang SC, Hashibe M, Vecchia CL, Parmigiani G, Edefonti V. Shared and study-specific dietary patterns and head and neck cancer risk in an international consortium. Epidemiology 2019; 30(1): 93-102.
5. cdc.gov [internet]: Centers for Disease Control and Prevention; c2009 [cited 2020 Jan 13]. Available from: https://www.cdc.gov/nchs/data/nhanes/nhanes_09_10/FCBS_f.pdf
6. fda.gov [internet]: Food and Drug Administration; c2014 [cited 2020 Jan 13]. Available from: https://www.fda.gov/media/96883/download
7. cancer.gov [internet]: National Center Institute; c2020 [cited 2020 May 25]. Available from: https://www.cancer.gov/publications/health-communication/pink-book.pdf
8. Ip EH, Marshall S, Vitolins M, et al. Measuring medical student attitudes and beliefs regarding patients who are obese. AAMC. 2013; 88(2): 282-289.
9. Leone LA, Beth D, Ickes SB, et al. Attitudes toward fruit and vegetable consumption and farmers’ market usage among low-income north carolinians. J Hunger Environ Nutr. 2012; 7(1): 64-76.
10.  cancer.gov [internet]: National Cancer Institute; c2005 [cited 2020 Feb 14]. Available from: https://cancercontrol.cancer.gov/sites/default/files/2020-06/theory.pdf
11. cdc.gov [internet]: Centers for Disease Control and Prevention; c2020 [cited 2020 June 19]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/index.html
12. cdc.gov [internet]: Centers for Disease Control and Prevention; c2020 [cited 2020 June 19]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/cases-in-us.html
13. clemson.edu [internet]: Clemson University; c2020 [cited 2020 July 1].

Available from: https://www.clemson.edu/degrees/food-science-and-human-nutrition

1. andersonfreeclinic.org [internet] The Anderson Free Clinic; c2020 [cited 2020 June 19]. Available from: https://andersonfreeclinic.org/for-patients-2/
2. aapor.org [internet]: American Institutes for Research; c2013 [cited 2020 June 19]. Available from: http://www.aapor.org/AAPOR_Main/media/AnnualMeetingProceedings/2013/Session_A-1-2-Noel.pdf
3. clemson.edu [internet]: Clemson University; c2020 [cited 2020 May 25]. Available from: http://media.clemson.edu/research/compliance/irb/verbal-consent-script.pdf
4. Lavin D, & Maynard DW. Standardization vs. rapport: Respondent laughter and interviewer reaction during telephone surveys. American Sociological Review. 2001; 66(3): 453-479.
5. Gathman CH, Maynard DW, Schaeffer NC. The respondents are all above average: Compliment sequences in a survey interview. Research on language and social interactions. 2008; 41(3): 271-301.
6. Kenneth M. Trust me, it is valid: Research validity in pharmacy education research. Currents in Pharmacy Teaching and Learning. 2016; 8(3): 391-400.
7. Zarah A, Enriquez-Maruland JA. Relationship between Dietary Habits, Food Attitudes and Food Security Status among Adults Living within the United States Three Months Post-Mandated Quarantine: A Cross-Sectional Study. Nutrients. 2020; 12(11): 3468.

PEER REVIEW

Not commissioned. Externally peer reviewed.