What we've learned from entrance & exit surveys in FLOSS and diversity outreach - AsheeshEdit
Why this session: the idea came after learning from the impressive feedback WikiHow got from a survey about their editor base.
What do we get from surveys, examples of surveysEdit
Surveys tend to bring information from users which organizers cannot imagine, given what they have to focus on.
Most responses arrive on day 1 (even within 5 minutes), then day 2 and 3.
Surveys are OK for free events.
Surveys to increase diversity of event participation.
What are the objectives of running surveys:
- tune the curriculum
- improve quality of the event
- identify what people want to learn for future events
- makes the event more credible (for entrance surveys)
Survey methodology and toolsEdit
Use of googledocs (Google spreadsheet app). Shorten the URL. Use a tool to get the e-mail when people answer.
Incentives to answer surveys:
- T-Shirt, goodies, ...
- give empowerement to users, build relationship
Importance of knowing the gender, in order to increase diversity in fere software in particular, also because you might get impressive data which you can then use. Note also that asking about gender is sometimes badly perceived.
In general, checked boxes give users the feeling that they are being put in a box (and may lead to them not answer the survey).
The order of questions in a survey is important. Example of maths exams, where moving the genre question at the end increases women success.
Limit the number of questions that appear in the survey.
Sometimes, an automated aknowledgement e-mail campaign can lead to interesting personal responses.
Open ended questions: it is a question of scale. If there are too many answers, they will not be exploitable.
Individual follow-up is always possible and useful.
Show the exit survey in advance, to prepare answers (and get better answers).
Surveys reveal what you care about: this could be positive or negative...
Publication of survey results:
- it is important for transparency reason
- aggregate some of the "too personal" answers (e.g. this instructor was awful)
- details of results published depends on whether the initial goal was for the community or for the organizer
- be clear from the beginning what will be aggregated and what won't
Survey problems and solutionsEdit
- How to use interesting results of surveys, in order to have an impact?
- How do you make sure that people who have left do answer?
- Provide time during the event to start answering the survey by displaying the URL on the screen.
- Use tweets during the event to display the link for the survey.
- Problem that people who answer surveys tend to be biased (really positive or really negative)
- Do a selective follow-up with the answers you obtain, to filter out the data.
- Participants are often reluctant to answer surveys, because they don't know what the data will be used for.
- Tell users what you are going to do with the survey results to reassure participants. Marketing usage versus improvement usage will lead to people answering or not the survey.
URL shortener: smarturl.it To be completed...