Application and Review Process
Contents
Application and Review Process¶
The application, review, and selection process have been developed to provide a fair opportunity for all the interested members to show their interest in attending our book dash events.
In this subchapter, we have provided details on the book dash’s application and reviewing process.
Call for applications¶
The book dash event is organised for up to 15 participants. The reason behind keeping it small is to allow the core team members to offer specific mentorship to the invited participants and make this an interactive, intimate, fun, and productive event.
To fairly make this selection, a call for applications is opened at least 3 months in advance so that interested members can share their skills, interest, and expected outcomes through a short application. The applications are kept open for a minimum of 4 weeks and the timeline is communicated clearly through our newsletters, Twitter feed, and application form.
In 150 words each, applicants respond to the following questions:
What could you contribute to the Turing Way?
What would you gain from being part of the Turing Way book dash?
These responses to the questions in the application form help us select a group of people who will be able to work effectively together and cover a broad set of possible contributions to the Turing Way. We don’t expect that the candidates might have previously contributed to the project or are familiar with our project infrastructure such as GitHub-based contribution. We look more for their ideas on what they think is important and helpful for others doing data science in the current scenario.
We then ask a few demographic questions. We want to make sure that the selected group of participants brings multiple skills, knowledge, and experience to The Turing Way. We aim to involve members from diverse groups, especially those who are traditionally excluded from data science based on gender, ethnicity, and career stage. Therefore, we ask our applicants to indicate if they consider themselves to be a member of one or multiple historically underrepresented groups. Here they can add any other information about themselves that they think the core team should know when reviewing their application (such as LGBTQ+, immigrant, disabled, parent, carer, POC members).
We seek to remove any barriers that might prevent our participants from attending the event. Hence, we use this form to assess what support we can offer, including funding requirements, to our participants in ensuring that they can comfortably attend the event. Applicants can mention their travel, accommodation, childcare, and accessibility requirements for their participation.
We continue to think of ways to ensure privacy and data protection. Currently, all these responses are securely handled under the GDPR (General Data Protection Regulation) law (learn about creating GDPR-proof surveys).
All our applicants must agree with the following GDPR statement when sending their application:
We need to store your e-mail address to contact you. All the details provided will be stored in a Google Drive that only core members of The Turing Way have access to. The servers are based in the USA. You can ask for your e-mail address and other identifiable information to be removed from our database at any time. If you have any questions please contact the team members by emailing turingway@gmail.com.
Reviewing process¶
Applications for The Turing Way book dashes are reviewed by at least 2 members of The Turing Way core team. A response form created by the organising members using the rubrics described below, the response from which are shared among all the reviewers during a designated meeting between the panel members. Scoring for each criterion is done between 1 to 3, where 3=criteria fully met, 2=criteria partially met, and 1=criteria not met. To avoid personal bias, each of these scores is clearly defined for every question in the review response form, which is explained below. This rubric aims to evaluate the application across multiple aspects and avoid any personal bias panel members may have.
This rubric is adapted from the Mozilla Open Leadership and the Open Life Science programs
Rubrics for scoring applications¶
The invited reviewers will rate the participants across multiple dimensions with scores of 1, 2, or 3.
We ask our reviewers to consider the following questions:
Has the applicant answered the application questions?
Does the applicant have a clear, feasible, and relevant idea of how they will contribute to the book dash? This scoring corresponds generally to reviews of “not ready”, “enthusiastic” and “clear” respectively:
Is the applicant enthusiastic about reproducibility and the aims of The Turing Way?
Based on their evaluation of the applications they will provide scores for different criteria that will be assessed as follows:
Applicants who score mostly 1s do not have a clear idea/topic ready for contribution or do not have a clear understanding of the overall goals of The Turing Way and book dash in general.
Applicants who score mostly 2s are enthusiastic if not wholly suitable for the project/event, for example, they may come with some ideas for a contribution that is currently not in the scope of the project/event.
Applicants who score mostly 3s are clearly ready, goal-oriented, interested in contributing to the project, and excited to learn from others at the event.
Table summarising our scoring rubric and criteria for each score:
(This scoring rubric is based on the Mozilla Open Leaders and Open Life Science selection criteria)
Sections |
Score 1 |
Score 2 |
Score 3 |
|---|---|---|---|
Readiness for the book dash event |
(not ready) Does not provide enough information or seems to misunderstand the nature of The Turing Way project and this event in general |
(enthusiastic) Seems to have a clear understanding of The Turing Way project and this event and brings along a specific content for contributions |
(clear) Seems to have a clear understanding of the Turing Way project and this event and a clear understanding of how they can contribute and collaborate with each other at this event |
Goals for the project contributions: |
(not ready) Shares vague or general ideas that are unrelated to The Turing Way project and this event, or no goals at all |
(enthusiastic) Shares clear, overly ambitious ideas for The Turing Way project and this event that can likely be refined in a collaboration cafe, or the day before the event during the brainstorm session |
(clear) Shares clear, achievable contribution/development ideas for this event that fits The Turing Way project and are likely to be achieved through the applicant’s participation |
Purpose of participation and what they will get out of the book dash: |
(not ready) Purposes for participation at the book dash seem almost entirely self-centered and about the applicant’s status, rather than about participating in The Turing Way community to develop the project |
(enthusiastic) Purposes for the participation at the book dash event are not completely clear from the application or are limited (even though useful), such as typo or bug fixing |
(clear) Purposes for participation at the book dash event are valuable in many ways and are likely to help the applicant to become an active contributor and take ownership of their work in The Turing Way project and in the broader ecosystem in their own rights |
Willingness to collaborate and contribute after the book dash: |
(not ready) Seems closed to collaborative ways of working or more interested in only one aspect of data science, research or related topic |
(enthusiastic) Seems excited to learn from others and The Turing Way project, but in a general way without much understanding of what those things mean yet |
(clear) Seems excited to collaborate with others and is motivated to contribute to The Turing Way community |
The following sections are aimed at collecting open-ended response by the reviewers:
We ask all reviewers to finish the review by describing the application, representation aspects of the applicant and the motivations for their scores to facilitate conversation during the selection panel discussion under these questions.
- Please highlight any diversity/minority groups this applicant belong to, that should be represented at this event
- Please provide 1-2 summary sentences about this application to facilitate discussion during the selection panel
Reviewers are also asked to disclose if any conflict of interest may have influenced their review leading to any bias in their decision.
- Please state if you have any conflict of interest with the proposal described in this application or do you know this applicant personally that may lead to a biased decision
Finally, reviewers are asked whether they would recommend an applicant. The available answers are “No”, “Yes”, “Unsure” or “Request for open discussion in the panel”. Candidates who have ratings of “No” or “Unsure” across all reviewers will be rejected. All other candidates will be discussed at the panel selection.