This paper came to my attention because of the extensive press
it received. The concept was also picked
up by places like the Atlantic. I gnawed on it for some time. Eventually, I contacted the authors. First about the most egregious problem. They corrected that in an amendment in ASQ. I then asked them to address the numerous other problems. At the end of the explanation, they provide a suggestion for correcting some errors and misrepresentations. What follows my original review and their response.
According to the Penn State News, this article finds that:
Corporate
CEOs who exhibit narcissistic personality traits are more likely to embrace
discontinuous or disruptive technologies than their less narcissistic
counterparts, according to research by Donald Hambrick of Penn State's Smeal
College of Business and colleagues from the University of Erlangen-Nuremberg
and IMD International.
Either I do not understand this paper, or the authors are extremely
confused, or it is poorly presented, or it is … misreported, or some mix of
these. As written, the model will not
run, the corrections presented do not correct, and obsolete practice is used to
understand mediation.
Some facts:
- The sample is of 33 firms over 28 years (possible 924 observations), but they get 435 or 521 depending on the table. That is likely because some firms merged etc.
- Their model: “For estimating the degree of adoption, we followed Allison and Waterman’s (2002) approach of using a Poisson specification (with dummies for each firm, to control for firm fixed effects) and then corrected standard errors for overdispersion by multiplying the standard errors by ‘‘the square root of the ratio of the Pearson chi-squared goodness-of-fit statistic to its degrees of freedom’’ (Allison and Waterman, 2002: 257).”
Indications the
actual statistical model will not run (the matrix will not invert) or will not
estimate meaningful coefficients.
- I am pretty sure one measure, Audience Engagement, is estimated at the year level and has no between firm variance. Yet in table 2, they claim that “Calendar year dummies and firm year dummies are included in all models.” The year dummies should be perfectly correlated with Audience Engagement. My guess is that STATA randomly kept AE and dropped one or more of the year dummies.
- They can’t have firm-year dummies for firm year data. /**A correction has since been published by ASQ.**/
- I don’t think it is possible to include both calendar year and firm-year dummies in the same model. Calendar year should be a linear function of firm year.
- They include a constant (Firm age_1980, that is the age of the firm in 1980) in the analysis. How can they include a constant in the analysis? If as they say they include dummies for each firm it will simply remove one of the year dummies. I guess STATA will run it and throw out the unusable variables, but I am not sure how to interpret it.
- “Because large and old firms may face bureaucratic momentum, but also possess more capabilities to adopt biotechnology (Ahuja, Lampert, and Tandon, 2008), we controlled for annual firm size (revenues t-1, deflated by the consumer price index) and firm age in 1980, except for tests ofH3 and H4, which were tested with ‘‘xtreg, f(ixed) e(ffects)’’
Misleading and
unjustified use of statistics.
- They do a “Heckman” correction by running a Z = BX + BI+ e, where Z is a continuous measure. They then use the prediction of E(Z) in the analysis and Z. Y = B*E(Z)+BZ+BX+e. The interpret the coefficient on Z. This is neither a Heckman nor an IV. I have consulted multiple statisticians about this. They can find no justification for this technique.
- “We controlled for endogeneity, or the possibility that narcissistic CEOs are attracted by or hired into specific situations and/or that narcissistic tendencies emerge under certain conditions. Following the two-step approach of Heckman (1979), we first regressed several antecedent and contemporaneous variables against our measure of CEO narcissism. The antecedents (measured in the year prior to the CEO’s start), included firm revenues, firm age, calendar year, and ROE, as these factors might affect narcissistic tendencies. The contemporaneous variables (measured in the CEO’s second tenure year), included CEO duality, CEO age, and CEO origin (a dummy indicating if the CEO was an outside hire, defined as having arrived at the firm within two years prior to becoming CEO). Two of these variables, firm age and calendar year, were significantly associated with CEO narcissism (p < .01) in correlations and multivariate regression.6 As the second step in the Heckman correction, we created a predicted narcissism score for each CEO, based on the significant variables from step one, and included it as a control in the main analyses.”
Common bad practice.
- They do a “mediation” analysis by dropping an endogenous variable into the regression.
I contacted both the authors and Jerry Davis (editor of ASQ). The authors have admitted to the
problems and submitted and amendment to ASQ about the problem created by the perfectly correlated dummy variables. What follows is their proposal for fixing other problems.
**
Dear Andrew:
We’re following up on your email of 28 May.
You are right – the continuous audience engagement
variable and year dummies should not have
been included in the same models, as they are perfectly
collinear.
Here again, in order to estimate the effect of the annual
value for audience engagement, Stata excluded two year dummies instead of just
one, both of which then served as reference points. Fortunately, as you pointed
out, Stata’s selection of the specific pair of reference years does not affect
the significance of the audience engagement moderator, given that the
interaction term is not multicollinear with the year dummies, so our
hypothesized results are not affected. (For our own education and satisfaction,
we have now conducted various experiments – dropping the linear audience
engagement variable and one calendar year dummy (instead of dropping two dummies),
as well as varying which pairs of year dummies are dropped – and the
hypothesized interaction effects remain fully robust.
Our suggestion would be to
modify our correction concerning this issue as follows:
In our analyses of
Hypotheses 1 and 2, which are reported in models 1-4 in Table 2 (p. 275), and
in the post-hoc analyses, which are summarized in the models 2-4 and 6- 8 in
the Appendix (p. 289), we erred by simultaneously including an invariant firm age
variable and firm dummies, as this caused perfect multicollinearity between
these two sets of variables. (Stata deals with this problem by dropping two
dummy variables, shifting the reference points for the fixed effects
coefficients.) Given that firm dummies themselves encompass effects from firm
age, we have re-run all respective regressions with the redundant firm age
variable excluded (while still retaining the firm dummies). Results of our
hypotheses tests remained identical (which is to be expected, as changing the
reference points only influences the coefficients and significance levels of
the dummy variables). Relatedly, we erred in including the continuous audience
engagement variable, which was measured annually, and year dummies in the same
models, as they are perfectly collinear. Here again, in order to estimate the
effect of the annual value for audience engagement, Stata excluded two year
dummies instead of just one, both of which then served as reference points.
However, the selection of the specific pair of reference years does not affect
the significance of the audience engagement moderator, given that the
interaction term is not multicollinear with the year dummies, so our hypothesized
results are not affected. Moreover, all results, including the hypothesized
interaction effects, remain the same if we drop the linear audience engagement
variable and one calendar year dummy (instead of dropping two dummies).
From your point for view, does that seem sufficient?
**
What do you think? Is
it sufficient?
