A hot mess: basketball coaches’ perceptions of ability versus actual performances of their athletes

Data analytics are an increasingly popular method for talent identification, and are used for a variety of decision making purposes, such as rotations and playing time. However, coaches often rely on their perceptions and experiences to identify talent and player attributes that are important to success. Therefore, the purposes of this study were to evaluate differences between coach perceptions of player ability against actual performances as well as to determine whether these perceptions differed as a head or assistant coach. Participants were six (two head; four assistant) college coaches who were asked to collectively identify the five most important attributes when evaluating a basketball player. Then, before the season began, all coaches were asked to independently score each of their athletes on these attributes using a 100mm Visual Analog Scale. These scores were compared to player performances during the season. Results were mixed, and while there were correlations between some player performance variables and coach perceptions, they varied wildly, and coaches’ perceptions of their athletes had little consistent correlation to their performances. Furthermore, there were few agreements between head coaches and their assistants or between assistants. Findings suggest that while coach perceptions and talent identification have their place, the use of data analytics in sports may provide additional support when making coaching decisions such as playing time. Therefore, coaches should recognize their own limitations of player talent and balance these “feelings” with statistical evidence.


INTRODUCTION
Coaches, at every level and in all sports, rely on their perceptions to make athlete-related decisions, such as roster selection, playing time, and tactical strategies. A perception represents an individual's ability to recognize an environmental stimulus using the senses (McFarland et al., 1999). In coaching, perceptions need to be recognized and accepted by the coach (Das & Phookun, 2013) if decision making is to be effective.
Unfortunately, perceptions can conflict with and cloud reality (Das & Phookun, 2013;Rickersten, 2012), especially when attitudes are taken into consideration (Pickens, 2005). Therefore, someone in an evaluative situation (e.g., coach, scout, manager) will likely have perceptions of an athlete's effectiveness based on a variety of circumstances, such as their prior performances, relationship with the player, and conversations with those associated with the player (e.g., former coaches, parents). These variables can influence how decisions are made. For example, in a recent documentary of Sunderland Football Club, owner Stewart Donald confessed to having purchased a striker for a record price, not because that was what he was valued at, but because he felt pressured into purchasing a striker before a transfer deadline. As days, then hours, then minutes ticked by, the player's perceived effectiveness and value to the club was progressively augmented by the urgency the owner felt to purchase a player prior to the transfer deadline and appease the club's fanbase (Pearlman & Turner, 2020).
Coaches are also required to make quick and often significant decisions based on their perceptions. Therefore, when judging performance or a situation, it is important that both coach and athlete share the same perceptions. For example, the feedback on a basketball player's jump shot technique provided by a coach might not be well-received or adopted by the player that does not perceive a problem exists. In many cases, such a discrepancy may exist; Oliver and Robins (1994) identified a bias in performance perceptions, noting that individuals perceived themselves more positively than their peers. Since self-perceptions and perceptions of others are often at odds, a single perspective may be an inadequate representation of reality (Vazire et al., 2008). Cook (2015) found discrepancies between coaches' and athletes' perceptions of performance, highlighting how important it is that coaches be aware of potential differences in performance perceptions when offering athletes constructive feedback. Although a lack of research exists on coaches' perceptions of athletes, numerous studies have examined athletes' perceptions of coaches. Smoll and Smith (1989) created a model of sport leadership behaviors to provide a framework for examining the cognitive and affective processes to help mediate an athlete's reaction to their coach's behavior. The variables of this model include: 1) situational factors (e.g., nature of the sport, level of competition, practice vs. game setting), 2) coach and athlete individual difference variables (e.g., age, sex, perceived coaching norms, goals/motives), and 3) the coach's perception of athletes' attitudes. They found coaching effectiveness to be mediated by an athlete's perception and recall as "leader effectiveness resides in both the behaviors of the leader and the eyes of the beholder" (p. 1544).
Turnman (2006) investigated how a coaching reward and power system can influence athletes' overall satisfaction with sport participation. Palmer (2013) investigated NCAA Division I female basketball players' perceptions of coaching behavior in relation to team cohesiveness and success. Findings indicated that players from both winning and losing teams selected the same top and bottom leadership functions of their coaches but rated the functions differently (Palmer, 2013).

Perceptions and analytics
Analytics and the use of statistics have become ubiquitous among professional sport teams, especially in the National Basketball Association (NBA; Fry, 2012). Analytics within the NBA has greatly impacted the way the sport is played and how player performance is measured. For example, it is used to study allocative and dynamic efficiency in decision making. Recent studies have demonstrated the growing use of athlete tracking data, which essentially tracks and numerically analyzes every aspect of an athlete's performance (Skinner & Guy, 2015). Player tracking data has been used with teams in the NBA to measure statistical improvements in player and team performance (Sampaio et al., 2015).
Tracking player movements in basketball has become important in overall performance of the team (Sarlis & Tjortjis, 2020). These tracking movements include statistics such as, speed, distance, player separation, and ball possession. Yet, much of the sport analytics produced and critiqued by statisticians can be overwhelming and there is inherent difficulty in understanding how to use this information effectively (Steinberg, 2015); coach perceptions may still hold some value.
Studies on coaching and analytics have been more prevalent than inquiry on coaching perceptions. In one study, game related statistics of basketball guards, forwards, and centers in three professional leagues were analyzed (Sampaio et al., 2006). Findings suggested that coaches perceived player's contributions to team performance differently according to the position of the player as either a guard, forward, or center rather than their contributions as a player in general. Therefore, there remains a challenge for coaches to balance analytical data, data that may fail to explain the story behind it (e.g., unusual circumstance, player injury, pressure moment), against their own perceptions that may be flawed.

Head coaches and assistant coaches
To date, there is no research evaluating the perceptions of head coaches against those of assistant coaches, which is a purpose of the present study. This is important, as with differing roles and responsibilities, each may view a given situation or athlete differently. According to Young (2020), head coaches in the NBA have a responsibility to establish and maintain a positive team culture, review and assess metrics associated with each game plan, allocate time for media events, participate in front office executive meetings, juggle demands from players about playing time and positions, and negotiate with a player's agent. Assistant coaches may be expected to establish and maintain a positive team culture, review and assess metrics associated with each game plan, allocate time for media events, participate in front office executive meetings, and also juggle demands from players about playing time and positions (Young, 2020).
Although most head coaches are responsible for making playing time decisions based on player performance, assistant coaches are often called upon to offer their input on player and team performance. As noted, with differing roles and responsibilities within the team, perceptions can differ. For example, perceptions by a basketball coaching staff can be influenced by a range of variables including physical characteristics such as height, weight, and athleticism, to more skill-based evaluations involving a player's specific offensive or defensive abilities (Turcott & Pifer, 2019). Furthermore, in stressful competitive games, assistant coaches may be placed in a situation where the power dynamic between head and assistant coach leaves them feeling incompetent (Zakrajsek et al., 2019). Consequently, an assistant coach may have a valuable observation or perception, but it may differ from the perception of the head coach and be ignored, unwanted, or withheld.

Study purpose
Perceptions are an important component of a coach's decision making. However, there is little research investigating this topic. In basketball and many other sports, analytics have become increasingly prevalent, yet a coach's perceptions that rely on immediate observations remain an important component of a coach's decision making. While some research exists on the perceptions of head coaches, little research exists to evaluate differences to assistant coaches. Therefore, the primary objective of this study was to determine whether head and assistant coaches' perceptions of their players correlated to their basketball performance. We hypothesized that perceptions of head coaches would be better predictors of basketball performance compared to assistant coaches. Our secondary aim was to determine the congruency between head and assistant coaches' perceptions of basketball performance. We hypothesized that the head coach and assistant coaches would have similar perceptions.

METHOD Participants
Participants (N = 6) consisted of members of one women's and one men's basketball coaching staff at a NCAA Division I institution (n = 2 head coaches, n = 4 assistants). The head women's coach was an African-American female with six years of NCAA Division I head coaching experience, one year as an interim NCAA women's head coach, four years as a NCAA Division I assistant coach and two years as a NCAA Division I graduate assistant coach. The first assistant women's coach was an African-American female with seven years of NCAA Division I associate head coaching experience, eight years of NCAA Division I assistant coaching experience and one year of NCAA Division I graduate assistant experience. The second assistant, a Caucasian female, had one year of NCAA Division I assistant coaching experience and two years as a NCAA Division II assistant coach. At the time of data collection, the first assistant had two years of experience working with the head coach; the second assistant had been a member of the coaching staff for one year.
The head men's coach was an African-American male with two years of NCAA Division I head coaching experience, five years of NCAA Division I associate head coaching experience, and had seven years as a NCAA Division I assistant coach. The men's first assistant coach was an African-American male a with one year of experience as a NCAA Division I assistant coach, two years of experience as a junior college assistant coach, and two years of experience as a graduate assistant at the NCAA Division I level. The second assistant, an African-American male, had two years of NCAA Division I assistant coaching experience with the current head coach. At the time of the investigation, the first assistant had one year of experience working with the head coach; the second assistant had been a member of the coaching staff for two years.

Instruments
Player Evaluation and the Visual Analog Scale (VAS). Head coaches and assistant coaches were collectively asked to identify the five most important attributes when evaluating a basketball player. After one hour of deliberation together, the following five attributes were selected: 1) Overall Basketball IQ (the ability of a player to impact the game), 2) Offensive IQ (a player's offensive ability), 3) Defensive IQ (a player's defensive ability), 4) Anticipation Ability (the ability of the player to anticipate the play), and 5) Passing Ability (the player's ability to pass the ball). Beyond the descriptions provided, coaches did not provide a more exact definition.
All coaches were asked to independently score each of their athletes on a 100mm Visual Analog Scale. The VAS is a scale that looks like a ruler, and had only two labels of "none" to "highest ever seen" at the extreme edges. Coaches were asked to make a perpendicular mark on the line that best represented their perception of each player's specific ability for each of the five dependent variables: Overall Basketball IQ, Offensive IQ, Defensive IQ, Anticipation Ability, and Passing ability. A VAS provides an accurate measure of subjective perceptions and has been used extensively in prior studies (e.g., Boolani, 2017;Jacobson, 2010;Wittwer, 2016). Two research assistants scored the VAS and entered the scores into SPSS. If a dispute occurred, the average of the two scores was utilized.
Performance Statistics. Player statistics were collected for an entire season and included: games started and played; total and average minutes played per game; field goals and three-point field goals attempted, made, and field goal percentage; free throws attempted and made; offensive, defensive, and total rebounds per game; personal fouls and ejections accrued; assists and assists per field goal attempt; turnovers and assists per turnover ratio; blocks, steals, and steals per assist ratio; points scored, scoring average per game, and Unadjusted Player Efficiency Rating (uPER). These season statistics were then averaged per 30 minutes of playing time for each athlete to standardize differences in playing time across the season. This process is similar to that used Klusemann et al. (2013), who used a 30-minute timeframe to standardize statistics across a basketball season.
Unadjusted Player Efficiency Rating (uPER). The uPER is a measure of basketball performance using a formula that calculates an individual performance rating per minute of playing time. This formula produces an unadjusted player efficiency rating (PER) and may be normalized and adjusted to the pace of play. Since college teams play against teams in multiple conferences, uPER was calculated, which has been used in multiple studies (Hwang, 2012;Rosenthal, 2014;Solieman, 2006;Zhang, 2011). uPER is unadjusted for the pace of the game whereas PER is adjusted by pace of the game. Therefore, because data extended beyond a team's conference schedule, and included teams from other conferences and competitive divisions, uPER was used.

Procedure
Following institutional review board approval, the head coaches and assistant coaches of a NCAA Division I team at the same university were approached and asked to participate in a study about basketball talent identification. Upon agreement, participants were asked to meet and determine the most important attributes of a basketball player. At the end of preseason practice, but prior to the commencement of competitions, coaches from both the men's and women's teams were asked to complete a 100mm Visual Analog Scale (VAS) for each of their players for the five different attributes they had identified as critical. The men's coaches rated 7 players and the women's coaches rated 11 players. Following this process, data were collected on players' performances throughout the season to allow for comparisons between coaches' perceptions and on-court performances.

Analysis
To determine whether head and assistant coaches' perceptions of their players accurately predicted individual basketball performance a pairwise Pearson's correlation was calculated between the head and assistant coaches' perceptions of players' attributes (i.e., Overall Basketball IQ, Offensive IQ, Defensive IQ, Anticipation Ability, and Passing Ability) and actual performance statistics. A secondary aim was to determine the congruency between head and assistant coaches' perceptions of basketball performance. This was tested using pairwise Pearson's correlations among head and assistant coaches' perceptions of their players. Analyses were conducted separately for the men's and women's teams and significance was set at p ≤ 0.05.

RESULTS
The purposes of this study were to evaluate differences between coach perceptions against actual performances as well as to determine whether these perceptions differed as a head or assistant coach. Results are broken down by men's and women's team as well as by whether the participant was a head or assistant coach.

Men's team
Tables 1-3 present the analyses for the men's team. Tables 1 and 2 present the correlations between the head coach and assistant coaches' perceptions of player attributes. Assistant coaches perceived player attributes in the same way as the head coach, as correlations were significant for almost all perceptions (Table 1). This was also true for comparison between assistant coaches, where the only correlation not significant was Offensive IQ (Table 2). Table 3 presents the correlation between players' real performance statistics and coaches' perceptions. Points, Defensive Rebounds, Blocks, and Steals had no significant correlations to any categories. However, Total Minutes (i.e., playing time) was determined based on the Head Coach's perception of Offensive IQ (r = 0.76) and Passing Ability (r = 0.77). Offensive Rebounds were negatively correlated to Defensive IQ for the head coach (r = −0.70), first assistant coach (r = −0.80), and second assistant coach (r = −0.70). Assists appeared to be important for all coaches for all categories except Anticipation Ability, but interestingly it was significantly, positively correlated to Defensive IQ by the head coach (r = 0.89) first assistant coach (r = 0.84), and second assistant coach (r = 0.77). Fouls were significantly, negatively correlated to Overall IQ by the assistant coaches (r = −0.80, −0.80) whereas it was tied to Offensive IQ by the head coach (r = −0.90).

Women's team
Findings were different for the coaches of the women's team, which are presented in Tables 4-6. Tables 4 and 5 present the correlations between the head coach and assistant coaches' perceptions of player attributes. There was little agreement overall; significant correlations were found between the head coach and first assistant coach for Overall IQ and Offensive IQ, and there was a significant correlation on Passing Ability with the second assistant coach. Assistant coaches had little agreement between themselves, with no significant correlations between their perceptions of players' attributes.     Table 6 presents the correlation between players' real performance statistics and coaches' perceptions. uPER was significantly, positively correlated to Passing Ability (r = 0.77) and Anticipation Ability (r = 0.73). In comparison to the coaches of the men's team, Total Minutes was not significantly correlated to any category, nor were Points, Offensive Rebounds, Turnovers, Defensive Rebounds, and Fouls. Assists were significantly, positively correlated to Passing Ability for the first assistant coach (r = 0.67) and Offensive IQ for the second assistant coach (r = 0.64). Interestingly, blocks were negatively correlated with Overall IQ for both the head coach (r = −0.40) and first assistant coach (r = −0.60). Steals were also negatively correlated to Anticipation Ability for the head coach (r = −0.60) but not for the assistant coaches.

DISCUSSION
The primary objective of this study was to determine whether head and assistant coaches' perceptions of their players correlated to their players' basketball performance. We hypothesized that perceptions of head coaches would be better predictors of basketball performance compared to assistant coaches. A secondary aim of the study was to determine the congruency between head and assistant coaches' perceptions of basketball performance. We hypothesized that the head coach and assistant coaches would have similar perceptions.

Perception versus actual performances
When considering our first hypothesis, results were mixed when considering the abilities perceived of players by the coaches and what transpired during the season. In addition, there was little congruence between the coaches of both men's team and women's teams. For example, while fouls were consistently negatively correlated to performance categories on the men's team (i.e., less fouls equated to a higher Overall IQ), they were positively correlated on the women's team. Other correlations appear counterintuitive; negative correlations between Defensive IQ and Blocks and Defensive Rebounds would indicate that perhaps coaches rely on perceptions more than they should. These findings are difficult to explain beyond the obvious finding that coaches struggled to identify the characteristics of their players that translated into performances across the season. Yet, this is also a telling finding, and highlights the value of analytics over the opinions and impressions of a coach. Given that individuals typically perceive themselves more positively than their peers (Oliver & Robins, 1994), it is likely that coaches value specific traits in their players more than others. For example, players on the men's team might expect to get more playing time if they are perceived to have good Passing Ability and a high Offensive IQ. On the women's team, higher scores on the uPER might gain the attention of the coaches.
Although we could not find any empirical writing on this topic, we suggest that a coach's previous coaching and playing experiences may have a role in this situation. For example, a head coach whose previous coaching experiences focused on defense may be more inclined to look for similar traits in their own athletes. Previous playing experiences may also be a factor. Most coaches have playing experience within the sport they coach (Ewing, 2019); therefore, they may be more inclined to look more favorably on a player who plays their position or possesses the same physical or psychological characteristics.

Head coaches versus assistant coaches
If a single perspective may be an inadequate representation of reality (Vazire et al., 2008), it benefits coaches to have assistants who can provide alternate suggestions and ideas. Assistant coaches in basketball can have many differing roles, including assisting with team selection or evaluating team performance (Young, 2020). Therefore, it is imperative that assistant coaches can identify attributes necessary for success just as much as head coaches.
In the present study, there was congruence between the men's team head coach and the assistant coaches on their players' attributes. Only Offensive IQ was not significantly correlated suggesting coaches evaluated their players similarly. The women's team coaches were much less in agreement. There were significant correlations between the head coach and first assistant coach Overall IQ and Offensive IQ and between the head coach and second assistant coach for Passing Ability. Correlations for Anticipation Ability and Defensive IQ were very low, suggesting how the head coach perceived these attributes was very different to the assistant coaches. Furthermore, correlations were low between the two assistant coaches.
Is it better to have strong agreement between coaches on player attributes or not? Congruence can lead to unity, with coaches evaluating players' attributes similarly. Yet, there may be value in having different perspectives. A coach that works exclusively with assistants who sees things "their way" may miss seeing new opportunities or ideas, and head coaches have reported hiring assistants with different perspectives and coaching styles (Rathwell et al., 2014). Therefore, while results from the present study suggest a lack of congruence between perceptions, this may in fact be helpful, as an assistant coach might moderate or balance the differing perspectives of a head coach or another assistant coach.

Application, limitations, and future research
The findings of the present study represent what a research never hopes to experience; data that makes little logical sense. Yet, nonsensical data can lead to conclusions, and we posit that based on our findings, coaches at all levels will struggle to identify player skillsets that will be revealed throughout a season. Therefore, coaches should be cautious about making position or playing time decisions too soon.
Further, these findings reaffirm that talent identification is difficult and coach perceptions are likely to exhibit biases, whether explicit or not. Wiseman and colleagues (2014) investigated whether hockey coaches and scouts could similarly rank player skills and found little agreement. Therefore, coaches and scouts required to recruit to their programs need to consider their own biases in their evaluations, and recognize that their evaluation may not translate into expected performances.
These findings lend further credence to the use of analytics in sports. Sarlis and Tjortjis (2020) recently suggested that data science and sport analytics are becoming increasingly common in basketball, lending support to decisions such as strengths and weaknesses in the game, evaluation of opponents, how to optimize performance indicators, team and player forecasting, team composition, and minimizing unpredictability. However, there must be balance. Were computers and mathematicians to serve as decision makers, many nuances that occur during sports will be missed. These are the nuances that might cause a poor performer to make the winning shot when it counts, for example. In their systematic review and meta-analysis, Roberts et al. (2019) suggested that coaches identify talent through their tacit knowledge and instinct, and struggle to articulate how they identify talent. Therefore, perhaps a combination of data and coach knowledge and feeling might provide the best solution for team selection and performance.
There are limitations to this study, which also provide avenues for future research. For example, this sample was limited to two collegiate basketball teams, and results may differ at other levels and sports. Further, the demographics of the coaches, including their playing and coaching experiences, will likely affect perceptions of their athletes. Therefore, investigating perceptions of coaches of different levels and experience may yield different results. Third, coach perceptions were acquired at the beginning of the season, and their players may have improved throughout the season thereby confounding results. Attaining coach perceptions throughout the season would help to alleviate this limitation.

CONCLUSION
To summarize, it is apparent that coaches' perceptions of their athletes have little consistent correlation to their performances. Yet, their perceptions of their players did appear have an impact on the amount of playing time the players received. Findings suggest that while coach perceptions and talent identification have their place, the use of data analytics in sports may provide additional support when making coaching decisions such as playing time. Therefore, coaches should recognize their own limitations at identifying player talent and balance these "feelings" with statistical evidence.