There has been an ongoing (and really rather bitter) argument over discrimination against women in the skeptical/atheist community – particularly over whether or not conferences are preferentially selecting old, white, male speakers. Arguably this could be expanded to include discrimination against youth and against different races, but the sexism issue is that which has been front-and-centre over the past year. The allegations have been that the organisers of various conferences (particularly TAM) have not been inclusive when considering female speakers and that this has contributed to an unwelcoming environment at skeptical conferences.
Plenty of people have made plenty of accusations and there have been assertions galore, but what do the data say? Well I can’t say that I have looked that hard, but I have only been able to find a couple of point estimates of gender and conference involvement from Jen McCreight and a brief note from Brian Thompson on the JREF’s Swift Blog. However, there seems to have been no rigorous collection, comparison, or analysis of the data.
Well I propose a bit of science! There are two hypotheses that we can test:
- Speakers at atheist/skeptical conferences are not representative of the diversity of the members of those communities [i.e. “we’re discriminating”].
- Speakers at atheist/skeptical conferences are becoming more representative of the members of those communities [i.e. “we’re getting better”].
Whenever people start talking about “diversity”, I instantly think of ecological questions rather than socio-cultural questions. It seems that the diversity in skepticism problem is one that is directly quantifiable using the suite of tools provided by the field of ecology.
The Biographical Data
The data that we need are information relating to the different speakers that have presented at conferences over the past couple of years. I propose that the following data be collected: sex, age, race, nationality, and education. This is analogous to the “trait” data in an ecological analysis.
The Attendance Records
Next we need to record which speakers were present at which conferences. I suggest the following conferences be included:
- TAM 1, 2, 3, 4, 5, 5.5, 6, 7, London, 8, London 2010, OZ, 9, and 2012 (14 conventions)
- Skepticon 1, 2, 3, 4, and 5 (5 conventions)
- NECSS 2009, 2010, 2011, and 2012 (4 conventions)
- Australian Skeptics Conventions (however many there are!)
I have made a start by collecting attendance and biographical data on most of the data for TAM 2012 speakers, and this can be seen in this Google Doc. This is analogous to the “community” data in ecology. If anyone else (SitP, local CFI chapters, Skepticamps, etc) wants to contribute data then that would be awesome, but I can’t go that far down the rabbit hole.
Is it Representative?
The wonderful thing about doing this analysis now is that the Atheist Census is currently running (and over 100,000 people have taken the short survey at the time of writing). While this isn’t necessarily going to give us a sample of skeptics, it will give us an idea of the freethought community more broadly from which we can infer the pool from which speakers could theoretically be drawn. We can quibble over the validity of this dataset and apply caveats accordingly but I see this as the best dataset available. The only drawback is that the Census doesn’t ask about race, but all other variables can be included in the analysis…
The Analysis
We test Hypothesis 1 (we are discriminating) by randomly sampling from the Atheist Census data to produce a lot (let’s say n=1000) different “speaker lists”. We then compare the diversity of the actual speaker list at different conferences to the simulated lists (which form a “null distribution”) to see if the actual lists are more or less diverse than what you would expect by chance. This is a standard procedure in statistical analysis.
We test Hypothesis 2 (we are getting better) by looking at time series plots of diversity across the different conferences. We might expect that there would not be a linear trend, as there were renewed efforts to enhance diversity prior to TAM9. I’ll consider more complex statistical models after eye-balling the data to see which is most appropriate.
I’m perfectly willing to be corrected on the methods, so I have posted the whole statistical code (in R, naturally) to a Google Doc. I’ve also uploaded two files showing examples of the biographical (actual data) and community data (mocked-up). Just to show you what these data look like in “ordination space”, here is a plot of some of 18 regular skeptical speakers. The similarity of speakers is shown by their distance (closer together=more similar). I’ve annotated the plot to show how the speakers group together (click to embiggen):
You’ll notice a couple of things: (i) the only non-white speaker is Leo Igwe who has his own little corner of the plot (not a good start for diversity…), (ii) Randi also has his own corner by virtue of seniority, and (iii) all the women are separated out on the left. The stereotypical skeptical speaker is the older, white male and they sit pretty well in the middle (Shermer, Krauss, Novella). A diverse group would have names in all the spaces, indicating that we had speakers not only representing extremes but also all aspects of the range of traits that we are measuring. A less diverse group would have a wide range of types of people, but big open gaps where intermediates weren’t present.
The Next Step
I need help to collate the data. The Google Doc that I posted contains about four hours’ of work for half of TAM 2012. Ideally the conference organisers (DJ et al.?) would supply these kinds of spreadsheets rather than me having to trawl archived websites, but I’m prepared to get this done the long way if needs be. However, if a dozen people get involved (and organised!) then we could get this knocked off in an afternoon! Also, if you have spoken at a skeptical con, feel free to enter (or correct!) your own biographical data. If you are interested in helping, leave a comment below and I’ll coordinate volunteers.
PS: I realise that some speakers might not want to reveal their ages. If that is the case then we will just guess based on photographs and online CVs (time since graduation, etc), so don’t be offended if I (or we if I get some help) get it wrong!
Katatrepsis 
I’m put off by the level of detail required by that doc. Time of day? Date? Duration? Session title? These are all extraneous, surely. Event, age, speaker name, country of origin, gender, ethnicity, education, all good.
Thanks for your feedback. I agree that they might not be central to the initial hypothesis testing, but there are a couple of important follow-up questions that will need to be asked, and having this detail to begin with will mean that nobody has to go back through all those programs! Here’s my justification:
Time of day: this is just for me to calculate the length of time that they were on stage.
Date: just for my records so I can locate the slot again. I can cut it.
Duration: I’ll calculate this myself from the times – I can cut it from the doc.
Session title: Informative because we can see who is talking about what. Are women just speaking on “women in skepticism” panels?
I’m in!
Your proposal seems quite problematic to me.
Firstly, it’s vulnerable to all the trappings of self-selected polls, online polls and it concerns a topic of which there appears to be a non-negligible portion of the population who have a political interest in its outcome.
Secondly, it appears to be methodologically flawed. From speakers at community conferences are not representative of the diversity of the members of that community, members of that community are being discriminated against does not follow. Your study does not account for confounding variables, such as variance between groups in their willingness to speak, their availability, their notoriety, their appeal to the community (celebrity status) and their representation within fields of interest to the community. Also race can be a problematic category.
Thirdly, I believe it is motivated by principal/ethical inconsistencies. If, for instance, skepticism or free-thought are fields which are entirely concerned with things like logical consistency, empiricism and evidence, things for which the sex (or race, etc) of the person invoking them is entirely irrelevant, then the sex of a person involved with these fields is entirely irrelevant to the field. If this is the case, then to seek sexual (or racial, etc) diversity within the skeptic or free-thought communities, is to attempt to structure that community according to criterion entirely irrelevant to it. You can’t have it both ways; either the sex, race or ethnicity of a person invoking skepticism or free-thought is relevant those fields (i.e. it effects their merit as a skeptic or freethinker) or it isn’t. If it isn’t, then the sexual, racial or ethnic composition of one of these communities is entirely irrelevant and there appears to be no relevant rational for preferring a sexually, racially or ethnically homogenous community to a diverse one, or vice versa.
Hi Al. Thanks for taking the time to provide feedback. Here are some thoughts on your points:
1. There is always bias in data, but we can apply caveats which soften conclusions in light of the limitations of those data. If we want to answer a question then we use what we have. Do you have a better suggestion? As for the politicisation of the outcome: this discussion has gone on for too long without any evidential basis. All I am trying to do is provide that basis. I am making everything transparent so that people can see what I have done.
2. I am testing a hypothesis that has been put forward before: that there are not as many speakers who are young, non-white and men. I can provide citations, if you like. If it is shown that there are fewer women/non-white/younger people on the speaker lists then we can start drilling down using the data that you suggest. The first pass is looking for a pattern. If you have data on the availability of different individuals then please do let me know. Also, I hope you realise how problematic (and circular) it is to say that speakers who are popular are those with greater “notoriety” and “celebrity status”.
3. There are two separate responses to this point. The first is that there is clear and unequivocal discrimination against women in a wide array of situations and so we should be conscious of that bias when we choose speakers for conferences. I am writing another post synthesising some of those carefully controlled experiments. My proposal would at least begin to provide data to evaluate whether this is happening in the skeptical movement. The result of such a bias would be the exclusion of deserving (potentially more deserving) individuals who have valuable contributions to make. The second point is that skeptical conferences do not just talk about string theory and evolution (they are NOT scientific conferences). There are many talks that are given by individuals who have unique experiences based on their particular background. In this regard, perhaps we should be seeking a diversity of speakers.
I understand your points, and at various times I have agreed with them. However, I do feel that the data and analysis presented here will provide a valid and useful insight into diversity in the skeptical community. I’d be interested to here your thoughts on my responses (perhaps one comment per point to make it easier to follow?).
1. I imagined you were aware of these issues, I felt they still needed to be put forward because they have the potential to severely decrease the reliability of the study’s findings. I am particularly exercised over the fact that the study will be conducted online, is self-selected and concerns politically motivated parties, as that assuredly increases the potential for fraud. Even if you were to restrict surveys to individual IP addresses, I’de imagine the prominence of computers and smart phones among this demographic could overcome that security measure. Perhaps, the census is large enough to overcome any of these potential biases, I am not a statistician, so I cannot say with any certainty.
2. (A) “I am testing a hypothesis that has been put forward before: that there are not as many speakers who are young, non-white and [wo]men” than what? Than those that are? Than would be representative of the larger community? I thought you were testing the hypothesis that “speakers at atheist/skeptical conferences are not representative of the diversity of the members of those communities”.
(B) It seems to me that your study may be laden with unwarranted assumptions and value judgements. After you state your first hypothesis you write [i.e. we’re discriminating], this appeared to me to be an inappropriate conclusion to draw if this hypothesis were confirmed. In your second hypothesis, you lay forth a value judgment in in your brackets [i.e. “we’re getting better”]. I would think this is a value judgment that it is better to have a composition of speakers at atheist/skeptical conferences that is more representative of the community’s constituents than not.
(C) It isn’t at all problematic or circular to say that speakers who are popular are those with greater notoriety or celebrity status, popular people are those with greater notoriety or celebrity status, that is what it means to be popular. Fame can be self perpetuating, it gets you public engagments which gain you notoriety which in turn gets you more public engagments. The point I was making was that if hypothesis 1 is confirmed, it may be explained by a non-representative distribution of notoriety or celebrity status among the pool of potential speakers. It is for this reason that a person like Richard Dawkins would be a more appealing choice of speaker than a less famous female, non-white or young speaker. However, as you clarify, you are only testing to see if their disparity in representation, not the causes for this disparity (should it exist). So, no conclusions should be drawn from your study, other than the presence or absence of said disparity.
3. (A) “The first is that there is clear and unequivocal discrimination against women in a wide array of situations and so we should be conscious of that bias when we choose speakers for conferences.”
Ok, I’m not sure as to what you are referring, it appears you are just performing some vague political posturing. If there were clear, and unequivocal discrimination against women at these conferences, you wouldn’t need a study to demonstrate it. It would be clear and unequivocal, such as a sexist, limiting clause in an organization’s charter. No such thing exists, so your point seems moot. There does appear to be clear and unequivocal discrimination within the charter of a chosen partner of one of the conventions though, but it isn’t against women. Skepticon has partnered with Secular Women, a non-profit which will deny grants to male applicants on the basis of their sex.
(B) “My proposal would at least begin to provide data to evaluate whether this is happening in the skeptical movement.”
No, it wouldn’t, and for the reasons I previously stated. Your data would only inform us of whether or not their is a disparity disparity between the composition of the speakers at skeptic conferences and the skeptic community at large.
(C) “The result of such a bias would be the exclusion of deserving (potentially more deserving) individuals who have valuable contributions to make.”
This would obtain even if there were no bias; to select a speaker from a pool, is, necessarily, to exclude other members of that pool.
(D) “The second point is that skeptical conferences do not just talk about string theory and evolution (they are NOT scientific conferences).”
Never did I state that skeptical conferences only concerned scientific topics.
(E) “There are many talks that are given by individuals who have unique experiences based on their particular background. In this regard, perhaps we should be seeking a diversity of speakers.”
Their background as what? Being a women? Not being white? What? Women and non-whites both, individually, make up more than 50% of the US population, so whatever experiences they may draw from these backgrounds, they most assuredly are not unique. Again, I fail to see how the color of one’s skin or the shape of one’s genitals has anything at all to do with one’s merit as a speaker at any of these conferences.
4. I noticed that you neglected to include SkepchickCon in your dataset, is there a reason for this?
I disagree with the premise of the experiment:
” Speakers at atheist/skeptical conferences are not representative of the diversity of the members of those communities [i.e. “we’re discriminating”].”
First, the events which can impact what speakers are at the conference is not being accounted for. To declare discrimination you would need to look at the number and demographics of what speakers were not only available to speak, but wanted to speak and had the necessary logistic support to speak and compare this to the list of actual speakers. The pool of speakers chosen from is limited to a subset of possible speakers. Further, the limitations in real life account for speakers who may be popular now, but were very low on the list of desirable speakers due to popularity at the time of the conference speaking invites. In addition to this, popularity or imfamy of given speakers would make them more desirable for conferences which are counting on such qualities for success and this would not amount to class discrimination except in the most obtuse sense of the word.
Lets have a quick look at the definition and uses for the word discrimination:
a : the act of discriminating
b : the process by which two stimuli differing in some aspect are responded to differently
2 : the quality or power of finely distinguishing
3 a : the act, practice, or an instance of discriminating categorically rather than individually
b : prejudiced or prejudicial outlook, action, or treatment
There are circumstances in which it would not be discrimination to only have older white guys. Without discussing those, we can admit that there is. Otherwise, it would be discrimination to have only women, or only black speakers etc. while calling it an open and inclusive conference.
== Speakers at atheist/skeptical conferences are becoming more representative of the members of those communities [i.e. “we’re getting better”].==
This second hypothesis is based on and worded as a foregone conclusion that there was some failure of an unacceptable nature in past conferences yet this conclusion does not yet have evidential support. Yes, there could be acceptable failures where the result is not desirable, but the evidence does not actually support class discrimination such as is being implied.
When and if the evidence shows a class discrimination in earlier conferences, then and only then can the evidence be used to say that discrimination is decreasing in respect of the area being monitored.
It does not need to be shown that there is discrimination to say that the diversity of speakers at atheist conventions has increased over time. Further, this change does not mean that there was discrimination. Correlation is not causation.
I think that to show discrimination you will have to show were the organizers actively turned away valid speakers because they are not popular old white guys.
Thanks for contributing to this little peer-review session! I agree that the models will be imperfect, but all models are imperfect. The question is “is it useful”, and I believe that it is! A quantification of the socio-cultural diversity of a conference is always going to be interesting, particularly if it varies between conferences. Now, why would you expect any of the variables that you have cited to vary between genders?
I would suggest that your particular definition of discrimination as “the act, practice, or an instance of discriminating categorically rather than individually” is that which is at the heart of the hypotheses that I have presented. There is overwhelming evidence for discrimination on the basis of gender across a whole range of fields, using controlled experiments where only gender varies. This indicates categorical judgements rather than individual judgements. We need to critically evaluate our own performance in this area, and all I am saying is “let’s have a look at the data that are available”. If you have better data then please let me know, otherwise we have to push on with what we have and soften our conclusions in light of the required caveats concerning data quality.
I agree that the way I have phrased the summary of the hypothesis (“we’re getting better”) pre-supposes a particular outcome, but this does not compromise the hypothesis itself. People can interpret the results however they wish, but the results will be interesting either way. What would a gradual increase in diversity mean for the skeptical movement? I’ll offer a few thoughts when I see the results, but others can feel free to chime in (even if it’s just to say “this is all nonsense”, so long as they can support their criticisms).
I agree that the best way to test this is experimentally. Why don’t you propose an experiment?
I don’t see too many ways to do this. The experiment should be to first record all available data. Write to organizers for each year and ask if they will surrender their list(s) for speaker selection. Check globally what speakers were making speaking engagements. That would include all authors of atheist materials etc. The efforts that have been made thus far make it easier (or should) to find lists like this as they seem to have given more spotlight to a more diverse group of eligible speakers.
Once you have the lists, find a way to rank them according to popularity, influence, and then demographics (gender, race, etc.) Now figure out availability for all known conferences. When you have a short list of available speakers for each conference, compare that to choices made. Lets see if the choices made were deliberately biased or merely a result of available choices.
So while you plot data as to who was chosen to speak you can also plot data as to who was available to speak. You should be able to take a stab at plotting data on each speaker and how many speaking engagements they have had in relation to when they are available to speak.
You should also plot speakers at national and global events in a different light than local or regional speaking engagements. This is one way to classify the speakers. To analyze this what you need is data, lots of data. The more data you can get the better. With only a small sampling of data there are no trends and everything is likely to be an outlier.
I say that while there may be many theories and hunches, gather the data and see what it says, have the gathering processes analyzed for failure points, and generally do nothing more than gather the data … let everyone do their own analytics.
I have a feeling this would be much more revealing than simply testing a hypothesis of your own.
I agree that your method makes sense. The problem would be the data collection. In particular, the eligible speakers are very difficult to identify because the only way they would be announced is if they were speaking (presumably?). Those potential speakers who aren’t speaking are effectively invisible.
Another potential experiment would be to submit papers to a judging panel of skeptics that were the same apart from the name. This is the classic experimental paradigm for testing for gender bias. We would design five (a random number!) different talk proposals and have male and female names for each (ten proposals in total). We would identify 30-40 (again, a random number, but the right ballpark) individuals within the skeptical community who have made these kinds of decisions on speaker selection. Each individual would be randomly given either the male or the female version (which differ only in the male or female name) of each of the five proposals and asked to rate out of 10. If there was a bias then you would expect to see higher ratings for male submissions.
I agree that the best thing is to make the data freely available (a very good point) and that is the reason for all the Google Docs. I will also conduct my own analysis and make public the statistical code and the results so people can take or leave what I find (at least they will know exactly what I have done).
If you make a good job of collecting the data, there are tons of people who would do their own analysis. Yes, it’s the hard part, but the most important part. My hopothesis is that you will have as much trouble finding available speakers for any given conference as the organizers did at the time.
Be certain to randomize the male and female names, so that you do not end up testing preferences for Johns or Jennifers.
the sad sad sad part is the lack of color. Now Dr.Tyson, good….and then we have people like Jamilla Bey, is she under “woman” or “color”? I know 2 speakers from TAM that claim Native American heritage, and I had a lovely talk with them about their heritage which they seemed to know quite a lot about. I like that SOMEONE is looking at this, and admitting no study will be perfect (which always encourages others to give it a try too). Too often we get “Well nothing has changed”. Has it or not? Is there an effort? I’d also say, is it truly a “negative” to have a doctorate? Should we try to have fewer speakers with doctorates? Oh well, Harriet Hall and Euginia Scott are not the same!! (even jokingly!)
Perhaps you can post data on who was invited as well as who spoke. It could be for instance that a very “diverse” group was invited but only old white males chose to accept the invitation to speak. The organizers may have the information on who was invited.
I am an economist so I naturally think of selection bias issues.