Sport science reporting in the real world

– Written by Martin Buchheit, France

On the 9 March 2013, Sir Alex Ferguson delivered in the Irish Times probably one of the most encouraging ever message for sport scientists in football: “Sports science, without question, is the biggest and most important change in my lifetime. It has moved the game onto another level that maybe we never dreamt of all those years ago. Sports Science has brought a whole new dimension to the game”. While such statements are gold for universities advertising sport sciences courses all over the world and for young students willing to embrace their carrier in elite clubs, the actual value of sport science may not always be rated as high in some elite clubs or federations¹. Having an impact on the training programme, as a sport scientist, is anything but easy¹. The way coaches and athletes understand, accept and use sport science is highly variable and unpredictable. The path leading to effective sport science support is a is a long and winding road, with frequent stops and constant redirections required. Historically, many mistakes have been made while we learned about the veracity and usefulness of our data and the best ways to report and implement sports science in the elite sports setting. Among the different components of effective sport science support, the three most important steps are likely the following:

1. Having an appropriate understanding and analysis of the data; i.e. using the right metrics and statistics. The first consideration is the choice of the best variables, i.e. those can be trusted in terms of validity and relativity and that can be useful to answer the questions that are actually asked by coaches and players. Second, working with relatively small numbers of athletes within a team setting as well as being unable to effectively control for many variables makes interpretation difficult with traditional analytical approaches such as Null Hypothesis Significance Testing (NHST, which includes ‘p values’ and ‘t-tests’ for example). Over the last decade or so, however, great strides have been made in understanding and reporting the effects we have on our athletes and more valid and relevant approaches exist which are much easier to clinically interpret². The modern practitioner working oblivious to these useful variables and analytical approaches could be considered incompetent, in my opinion, whereas a practitioner aware of these approaches but clinging to the past borders on disingenuous.
2. Offering attractive and informative reports via improved data presentation/visualisation. Effectiveness in this step depends likely more on artistic skills and a creative mind than proper scientific knowledge and this is often overlooked in sport sciences programme. Day-to-day trials and errors are likely key in the search of the optimal data visualisation strategies.
3. Having appropriate communication skills and personal attitude to efficiently deliver these data and reports to coaches and athletes. This step is without doubt the most important of the process; there is however no training offered at universities for this. Nothing replaces experience, high personal standards and humility at this stage, which is generally developed over time.

The following sections will detail each of these three components.

COLLECTING AND UNDERSTANDING THE (RIGHT) DATA

The first important step to build a successful sport science system is to choose and work with the right data³. With the exponential rise in (micro) technology, collecting data from athletes has never been so easy. For every training session it is relatively easy to fully characterise both the external (e.g. tracking systems, encoders, force plates) and the internal load (e.g. heart rate, muscle oxygenation, sweat rate) placed on each athlete. However, technology per se might not be the solution; the foundations of successful sport science support are probably laid on the pitch first, when practitioners select the type of data that may help them to answer the questions that coaches and athletes have actually asked, in the way they collect these data, how they understand the limitations of each variable and how they analyse, report and utilise all this information³. While validity/reliability studies are important in the search of the best variables, their practical usefulness should also not be overlooked, i.e. their ability to be used to impact on the training programme. This relates to ‘interesting vs important’ types of data. For example, measurement of maximal oxygen uptake vs maximal aerobic speed; only the latter can be used for training prescription.

Statistics are probably one of the most important aspects of sport science when it comes to using data to make decisions. Unfortunately, the statistical proficiency of most practitioners in the field is often insufficient to maximise the use of their data and in turn, impact meaningfully on training programmes. One of the main reasons for practitioners’ lack of ‘statistical efficiency’ is that statistical lectures at university have, to date, exclusively sung the praises of NHST, which is:

• Not appropriate to answer the types of questions that arise from the field: as detailed in Table 1, the magnitude of an effect is what matters the most to practitioners – P values don’t inform this⁴.
• Not appropriate to assess individuals, which is the core of elite athlete monitoring. In fact, conventional statistics allow analysis of population-based responses only (Table 1)⁴.

As a valid alternative to NHST, clear analytical advances can be reached using magnitude-based inferences (MBI, Table 1). This ‘new’ statistical approach, driven largely by Will G. Hopkins’ efforts over the past 15 years, has changed my life, both as an academic and practitioner in elite sport¹¹. I personally hope that MBI is influential with other scientists, as it has been to me. While the debate will likely continue, MBI is today a well-established analytical approach in sports science and in other fields, particularly clinical medicine where practical/clinical significance often takes priority over statistical significance⁴.

MBI is based on two simple concepts:

1. Changes/differences in any variable are systematically compared to a typical threshold representative of a smallest important or meaningful change (later to be termed the smallest worthwhile change, SWC¹²).
   1. Why? Not all changes are worthwhile/meaningful. It is the magnitude of the change/difference that matters first: ‘is the change larger/greater than the SWC? If yes, how many times greater?’ In this context, change/differences of 1x, 3x, 6x and 10x SWC can be considered as small, moderate, large and very large, respectively⁴.
   2. How? The most appropriate method to define it is however variable-dependent, which forces researchers to adopt a conscious process when analysing their data. “NHST is easy, but misleading. MBI is hard, but honest” (W.G. Hopkins, personal communication)⁴. Recommendations to calculate the SWC are provided in Table 2.
2. Instead of a classic ‘yes or no’ type response (NHST), the probabilities for these changes/differences to be ‘real’ (greater than the SWC) are reported.
   1. More precisely: chances are reported both quantitatively (e.g. 75/25/0 for percentage chances of greater/similar/smaller magnitude than the SWC) and qualitatively (e.g. possibly, likely, very likely – Figures 1 and 2, and Table 3).
   2. How? These percentage chances and associated qualitative interpretations are generally set a priori (e.g. <1%, almost certainly not; 1 to 5%, very unlikely; 5 to 25%, probably not; 25 to 75%, possible; 75 to 95%, likely; 95 to 99, very likely; >99%, almost certain)
   3. Practically: these percentage chances can be obtained with only a few copy and paste manoeuvres using a specifically-designed spreadsheet freely available online^13,14. Final decisions can then be translated into plain language when chatting with coaches: ‘This attacker has very likely increased his sprinting speed. The magnitude of improvement should be enough for him to win a few more balls during matches.’

PRESENTING THE DATA

Similar to the aphorism that all roads lead to (and therefore from) Rome, the same data and results set can be presented in many ways (Figure 3). Once the relevant questions have been identified, the best variables have been selected and the appropriate statistics applied, the greatest challenge for sport scientists is to find the most efficient type of data visualisation and reporting to get their message across. Several considerations to optimise tables, graphs and content presentation are discussed below and illustrated in Table 3 and Figure 3.

1. Reports should be as simple and as informative as possible (‘simple but powerful’):
   1. Limited to a few ‘important’ variables (those that can be used to answer the questions that coaches and athletes have actually asked and can have an impact on the programme).
   2. Extra decimals and ‘noise’ removed for clarity (Table 3).
   3. All text written horizontally for readability (Figure 3b).
   4. Labels added to graphs so that exact values can be seen too (graph for patterns, numbers for details, if required) (Figure 3b).
   5. Meaningful changes or differences highlighted to be seen at a glance (Figure 2) – with different possible levels of data analysis. Microsoft Excel’s conditional formatting depicting MBI is a useful example (Table 2).
   6. Including error bars where possible to acknowledge uncertainty (typical error of the measurement and confidence intervals for individual and average values, respectively) (Figure 2 and 3).
   7. Using advanced visualisation tools such as Tableau or Microsoft BI. Although these require some training, they may be helpful to create aesthetically pleasing and advanced reports that may be more likely to catch coaches’ and athletes’ attention.
2. Format of the message should match with coach and athlete expectations, preferences and habits (which is linked to the search of the best delivery path, see below):
   1. Visual vs verbal information.
   2. Paper vs digital reports.
   3. Quantitative vs qualitative interpretation.
   4. Tables vs graphs (and types of graphs, e.g. bars vs radars etc.)

DELIVERING THE DATA

This last section is definitively less scientific than the previous two: it rather reflects personal views based on experiences and discussions with peers in the ‘industry’. These ideas were recently summarised in an editorial for the International Journal of Sports Physiology and Performance¹. While delivering the data is only one of the three steps highlighted in the present paper, it may be the most important. If sport scientists can’t communicate with the coach, if they can’t create interest and interactions with the coaches and players, then they obviously won’t manage the get the message through (i.e. deliver) and their fancy reports with high quality stats will end up in the bin. The authors conclude: “Masters’ degrees and PhD qualifications often are of little benefit in the quest of creating such a collaborative and productive environment. Understanding the specific codes of a sport or a very specific community of athletes takes many years. Having the respect and trust from high-profile athletes is often more a matter of personality and behaviour than scientific knowledge and skills. As described by the fantastic Dave Martin, we, sport scientists (monkeys) and coaches and athletes (felines and big cats) don’t belong to the same species. We have different expectations, behave differently and tend to make our decisions based on evidence and facts, while they rely on feelings and experience. Creating these links, building these bridges requires time and effort. Since the majority of coaches, supporting staff and athletes often don’t know what to expect from scientific support at the club, it is only by sitting right next to them during training sessions and team debriefs, by sharing meals and coffees, being with them in the 'trenches' that sport scientists can appreciate what coaches and athletes may find useful and which information they rely on to make their decisions¹.” Leaving a report on a desk or a bench in not impactful; it is the conversation that makes the data meaningful and that can only occur once a relationship has been developed. Also, while having a strong character is often compulsory to survive in most places, open mindedness, humility and a form of kindness are probably some of the most important personality traits to develop to make an impact in this world. With these personal and social engagement skills in mind, it is not surprising that the majority of the most renowned researchers, sports scientists and performance managers to date have, in parallel to their academic journeys, exposed themselves deeply to the elite sport culture, either directly (as athletes) or indirectly (as coaches)¹. Only those may have the ability to properly deliver data reports and influence decisions accordingly.

CONCLUSION

The value and importance of sport science varies greatly between elite clubs and federations. Among the different components of effective sport science support, the three most important elements are likely the following:

1. Appropriate understanding and analysis of the data; i.e. using the most important and useful metrics only and using magnitude-based inferences as statistics. In fact, traditional null hypothesis significance testing (P values) is neither appropriate to answer the types of questions that arise from the field (i.e. assess magnitude of effects and examine small sample sizes) nor to assess changes in individual performances.
2. Attractive and informative reports via improved data presentation/visualisation (‘simple but powerful’).
3. Appropriate communication skills and personality traits that help to deliver data and reports to coaches and athletes. Developing such an individual profile requires time, effort and most importantly, humility.

Martin Buchheit Ph.D.

Head of Performance

Paris Saint Germain Football Club

Paris, France

Contact: mbuchheit@psg.fr

References

1. Buchheit M. Chasing the 0.2. Int J Sports Physiol Perform 2016; 11:417-418.
2. Batterham AM, Hopkins WG. Making meaningful inferences about magnitudes. Int J Sports Physiol Perform 2006; 1:50-57.
3. Buchheit M, Simpson B. Player tracking technology: half-full or half-empty glass? . Int J Sports Physiol Perform 2016 [In press].
4. Buchheit M. The numbers will love you back in return – I promise. Int J Sports Physiol Perform 2016; 11:551-554.
5. McCormack J, Vandermeer B, Allan GM. How confidence intervals become confusion intervals. BMC Med Res Methodol 2013; 13:134.
6. McGuigan MR, Cormack SJ, Gill ND. Strength and power profiling of athletes: selecting tests and how to use the information for program design. Strength Cond J 2013; 35:7-14.
7. Pettitt RW. The standard difference score: a new statistic for evaluating strength and conditioning programs. J Strength Cond Res 2010; 24:287-291.
8. Al Haddad H, Simpson BM, Buchheit M. Monitoring changes in jump and sprint performance: best or average values? Int J Sports Physiol Perform 2015; 10:931-934.
9. Hopkins WG. How to interpret changes in an athletic performance test. Sportscience 2004; 8:1-7.
10. Cohen J. Things I have learned (so far). Am Psychol 1994; 45:1304-1312.
11. Buchheit M. Any Comments? 2013. Available from: www.herearemycomments.wordpress.com/. [Accessed 16 March 2016].
12. Hopkins WG. Statistical vs clinical or practical significance [Powerpoint presentation]. Sportscience 2002; 6. Available from: www.sportsci.org/jour/0201/Statistical_vs_clinical.ppt
13. Hopkins WG. Precision of the estimate of a subject's true value [Excel spreadsheet]. In: Internet Society for Sport Science. Sportscience 2000. Available from: www.sportsci.org/resource/stats/xprecisionsubject.xls2000 [Accessed November 2016].
14. Hopkins WG. A spreadsheet for deriving a confidence interval, mechanistic inference and clinical inference from a P value. Sportscience 2007; 11:16-20. Available from: www.sportsci.org/2007/wghinf.htm [Accessed November 2016].
15. Buchheit M, Allen A, Poon TK, Modonutti M, Gregson W, Di Salvo V. Integrating different tracking systems in football: multiple camera semi-automatic system, local position measurement and GPS technologies. J Sports Sci 2014; 32:1844-1857.
16. Buchheit M, Morgan W, Wallace J, Bode M, Poulos N. Physiological, psychometric, and performance effects of the Christmas break in Australian football. Int J Sports Physiol Perform 2015; 10:120-123.
17. Haugen T, Buchheit M. Sprint running performance monitoring: methodological and practical considerations. Sports Med 2016; 46:641-656.
18. Hopkins WG, Marshall SW, Batterham AM, Hanin J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc 2009; 41:3-13.
19. Buchheit M. Monitoring training status with HR measures: do all roads lead to Rome? Front Physiol 2014; 27:73.