Calibration / Validation of the YAP
The Youth Activity Profile is a calibrated report tool that provides estimates of physical activity (PA) and sedentary behavior (SB) based on validated prediction algorithms. The work has involved a 10 year progression of research but the primary calibration and validation work is described in a published paper by Welk et al. 2021. 

The software incorporates the published algorithms to enable users to obtain accurate group estimates of physical activity for school-based applications or for research. However, it is important to note that the precision of the algorithms may vary depending on the nature of school setting (e.g. urban / rural) or the location since there may be differences in how youth respond to items in different places. Contrary to popular belief, there is no single study that can 'validate' an instrument. A published paper can provide evidence of validity but ongoing research is needed to test the algorithms in different settings and populations.

A number of other colleagues and collaborators have used the YAP in various school-based applications (see papers by researchers affiliated with the SWITCH project and the Wellscapes project). Planned work is aimed at develop spatial measurement error models that would capture the variability in different regions and cultures more directly. It is also important to point out that studies have also tested the reliability, feasibility and validity of the YAP in international applications (see Fairclough et al and . Work is in progress to build an international collaboration to directly evaluate the utility in different countries for cross-cultural applications.

References

  • Kellstedt, D. K., Essay, A. M., Schenkelberg, M. A., Rosen, M. S., Von Seggern, M. J., Idoate, R., Welk, G. J., Rosenkranz R.R., & Dzewaltowski, D. A. (2022). The COVID-19 pandemic and changes in children’s physical activity in a rural US community: a mixed methods study. BMJ open, 12(10), e062987. doi: 10.1136/bmjopen-2022-062987
  • Kellstedt DK, Essay AM, Schenkelberg MA, Rosen MS, Von Seggern MJ, Idoate R, Welk GJ, Rosenkranz RR, Dzewaltowski DA. Disruptive events and system change: the COVID-19 pandemic and changes in children's physical activity in a rural community. The Journal of Rural Health. 2022.
  • Welk, G.J., Saint-Maurice, P.F., Dixon, P.M., Hibbing, P.R., Bai, Y., McLoughlin, G.M., Pereira da Silva, M. (2021). Calibration of the Online Youth Activity Profile Assessment for School-Based Applications. Journal for the Measurement of Physical Behavior
  • Schenkelberg, M. A., Rosen, M. S., Bavari, A. E., Norgelas, S. J., Rosenkranz, R. R., Welk, G. J., & Dzewaltowski, D. A. (2021). A protocol for coordinating rural community stakeholders to implement whole-of-community youth physical activity surveillance through school systems. Preventive Medicine Reports, 101536. PMID: 34976611. DOI: 10.1016/j.pmedr.2021.101536.
  • Pfledderer, C. D., Burns, R. D., Byun, W., Carson, R. L., Welk, G. J., & Brusseau, T. A. (2021). School‐based physical activity interventions in rural and urban/suburban communities: A systematic review and meta‐analysis. Obesity Reviews, e13265.
  • Kellstedt, D. K., Schenkelberg, M. A., Von Seggern, M. J., Rosenkranz, R. R., Welk, G. J., High, R., & Dzewaltowski, D. A. (2021). Youth sport participation and physical activity in rural communities. Archives of public health, 79(1), 1-8.
  • Kellstedt, D. K., Schenkelberg, M. A., Welk, G. J., Rosenkranz, R. R., Idoate, R., Ramos, A. K., ... & Dzewaltowski, D. A. (2021). Rural community systems: Youth physical activity promotion through community collaboration. Preventive Medicine Reports, 23, 101486.
  • Segura-Díaz, J. M., Barranco-Ruiz, Y., Saucedo-Araujo, R. G., Aranda-Balboa, M. J., Cadenas-Sanchez, C., Migueles, J. H., Saint-Maurice, P.F., Ortega, F.B, Welk, G.J.,  Herrador-Colmenero, M. & Chillón, P. (2020). Feasibility and reliability of the Spanish version of the Youth Activity Profile questionnaire (YAP-Spain) in children and adolescents. Journal of Sports Sciences, 1-7. DOI: https://doi.org/10.1080/02640414.2020.1847488
  • Fairclough, S. J., Christian, D. L., Saint-Maurice, P. F., Hibbing, P. R., Noonan, R. J., Welk, G. J., ... & Boddy, L. M. (2019). Calibration and Validation of the Youth Activity Profile as a Physical Activity and Sedentary Behaviour Surveillance Tool for English Youth. International Journal of Environmental Research and Public Health, 16(19), 3711. PMID: 31581617. DOI: 10.3390/ijerph16193711
  • Welk, G. J., Saint-Maurice, P. F., Kim, Y., Ellingson, L. D., Hibbing, P., Wolff-Hughes, D. L., & Perna, F. M. (2017). Understanding and interpreting error in physical activity data: insights from the FLASHE study. American journal of preventive medicine, 52(6), 836-838.
  • Saint-Maurice, P. F., Kim, Y., Hibbing, P., Oh, A. Y., Perna, F. M., & Welk, G. J. (2017). Calibration and validation of the Youth Activity Profile: the FLASHE study. American Journal of Preventive Medicine, 52(6), 880-887.
  • Saint-Maurice, P. F., Welk, G. J., Bartee, R. T., & Heelan, K. (2016). Calibration of context-specific survey items to assess youth physical activity behaviour. Journal of Sports Sciences, June 1-7.
  • Saint-Maurice, P. F., & Welk, G. J. (2015). Validity and Calibration of the Youth Activity Profile. PloS one, 10(12).

Segmented Approach to Estimating PA Levels in Youth

 A unique aspect of the YAP is that the PA items (#1 to #10) are individually calibrated. This provides estimates of the time spent in PA for different segments of the day. Separate calculations are provided for the 5 items capturing PA in school’ (items 1-5), the 3 items capturing PA at home (items 6-8) and the 2 items capturing PA on weekends (items 9-10). The algorithms predict the % time spent in PA so the calculated time is based on standard time segments for each item.  The calculated PA estimates are added together and averaged to provided estimates of PA on weekdays (combination of School PA and Home PA), PA on weekends (combination of Saturday / Sunday) and a weighted average based on 5 weekdays and 2 weekends. A separate composite calibration is used for estimating sedentary behavior.
Picture
Foundational Research on the YAP
The foundational calibration research for the original YAP calibrartion was carried out by Dr. Pedro Saint-Maurice as part of dissertation research in collaboration with Dr. Greg Welk in the Department of Kinesiology. The work demonstrated that raw scores from the YAP can be re-scored to estimate minutes of MVPA. This section presents some images from the original calibration work with the YAP. The figures are from different papers and projects to show how self-reported activity was consistently associated with measured activity levels obtained from accelerometers. Details from these original studies are provided below for context but the software currently uses the versions of the algorithms in the most recent paper on the online tool.
Figure 1 (below) illustrates the relationship between accelerometer activity levels (recorded %MVPA) and predicted activity levels (predicted %MVPA) in a sample of 103 participants (grades 4 through 12) from 9 schools (during the 2009–2010 school year). The solid line represents the best fit with the respective 95% confidence interval for the mean predicted values (dashed lines). ​The final model explained 40% of the variability in Recorded % MVPA. See Saint-Maurice et al, 2014.

FIGURE 1

Figure 2 (below)illustrates the relation between predicted and recorded minutes of MVPA in a sample of 45 youth (4th through 12th grades). The solid line represents the best fit line and the dashed lines represent the 95% confidence bounds about the best fit line. On average, the PAQ calibration equation produced similar accelerometer estimates of total minutes of MVPA (Mean diff. = 25.3 ± 18.1 minutes (t (44) = 1.40, p = .17).
See Saint-Maurice et al, 2017. 

FIGURE 2

Figure 3 (below) illustrates the average estimates obtained from 161 youth (4th through 12th grades) of recorded (SWA) and predicted MVPA (YAP) for weekly activity at school and at out-of-school (segmented by window of activity). The plot shows that YAP individual items approximated measured activtiy for the respective time segments. Follow-up analyses demonstrated that YAP school and out-of-school activity were within 15% and 20% of SWA measured activity values for the respective segments. See Saint-Maurice & Welk, 2015.

FIGURE 3

FIGURE 4

Figure 4 (Below) illustrates group-level agreement for school (n=51), out-of-school (n=51), and weekend (n=39) activity using a 10% and 20% equivalence zones. The solid black line bar represents the confidence interval for predicted YAP MVPA values while the dashed and the rectangular lines represent the 10% and 20% equivalence zones based on GT3X+ MVPA values obtained from the  wrist location, respectively. The algorithms tested were developed as part of the FLASHE study using a sample of ~119 adolescents (12-18yrs).
​See Saint-Maurice et al, 2017.  
Figure 5 (below) illustrates the relation between measured and predicted minutes of time spent in sedentary. The line of best fit (solid black line) and respective 95% confidence interval (dashed black line) are provided. The predicted values illustrated above were validated on 161 children and adolescents (4th-12th grades). The algorithms tested were developed using a sample of 291 4th-12th grades.
​See​ Saint-Maurice & Welk, 2015. 

FIGURE 5