Prolific will link participants to our experiments, which will be hosted on a different website (i.e. Gorilla, Pavlovia). Participants will complete the consent form, demographics form, and experiment on this other website. Once they finish, you will have to either accept or decline their data before they are paid via Prolific.

​

mTurk

Amazon’s Mechanical Turk (MTurk) allows for participant recruitment, payment, and experiment hosting all in one. Experimenters will also be able to filter subjects by demographic information and will have to accept or decline data before participants are paid. MTurk participants are assigned a ‘Worker ID’ which serves a similar function as the ‘Prolific ID’.

​

Check the IRB protocol(s) you are on for IRB-approved recruitment materials. Use these recruitment materials for your online experiment. Make sure to state the exact amount that your participants will be paid. Additionally, you can name your study whatever you see fit.

​

Make sure to check the eligibility criteria on your IRB protocol. Typically, our participants must be (1) 18 years or older, (2) located in the US, and (3) able to read and write in English. We often also have an upper age limit; check your IRB protocol to be sure and implement that upper age limit when recruiting online.

​

​

Experiment Hosting Websites

For hosting the experiment itself online, you have a few options:

​

Gorilla

Gorilla is an experiment builder that hosts the experiment for you, has a GUI-based and code-based (JavaScript) builder, and will export CSV data for you. Most experiments are able to be built through the GUI builder relatively easily. Gorilla works directly with Prolific and you will have to accept or decline the data on both websites.

​

Pavlovia

Pavlovia is an experiment builder that works with PsychoPy3. In PsychoPy, there is an option to convert your experiment into JavaScript but this process is not perfect (PsychoJS is the online counterpart of the PsychoPy Python library). This process seems to work for the most part, being able to convert most aspects of the experiment, but there will likely be some minor bugs to troubleshoot. For instance, one known issue is not being able to control the mouse position on the screen—this just isn’t possible in browser-based experiment. Alternatively, you can code the entire experiment in JavaScript. Pavlovia supports jsPysch and lab.js which are different JavaScript based experiment builders. Pavlovia also works directly with Prolific.

​

MTurk

MTurk allows for participant recruitment, payment, and experiment hosting. MTurk experiments can be coded in JavaScript and hosted using Amazon’s Web Services (AWS). The experiments hosted on this site are called Human Intelligence Tasks (HITs). The tasks normally hosted on this site are not very interactive, and creating experiments in which trials are adaptive may be difficult. As an alternative, MTurk can also link your participants to other websites (Gorilla, Pavlovia) or you can use psiTurk for more complex experiments. Here are some other basic tips on how to get the most out of MTurk.

 

 

Paying Participants & Other Fees

Check your IRB protocol for the approved pay range. Pay your participants within this range. Make sure this pay range is above the minimum that Prolific and MTurk require. If we are currently paying below the minimum, let Mariam know so the IRB protocols can be modified to meet the new minimum.

Prior to running subjects, you will need Mariam to put money into the website platform that you are using.

 

You will need to have enough money in the account to cover (1) the subject payment fees, and (2) the website fees.

​

• For Prolific, the service fees depend on the quantity of participants you want to run, this information won’t be known until the experiment is set up.

• For Gorilla, there is a $1.08 fee per participant. Again, participants are not paid on Gorilla.

• For Pavlovia, there is a £0.20 fee per participant. Again, participants are not paid on Pavlovia.

• For MTurk, pricing details depend on the study. They provide a cost calculator.

 

Here is some info on getting reimbursed for these fees. Because Mariam will put in money for you, she will get reimbursed — but she could use your help in getting this information together.

 

If using Prolific, please provide Mariam with the following information:

• The IRB number and name of study

• List of Participants provided by Prolific

• Proof of Payment

• The amount paid out to subject recruitment for that particular reimbursement (i.e., the amount you have receipts for)

 

If using MTURK for recruitment payment, please provide Mariam with the following information:

• The IRB number and name of study

• The number of “HITS” (the amount Mariam will be reimbursed for)

• A copy of proof of payment from AMAZON

• A copy of the MTURK payments (spreadsheet)

 

 

Experiment Building - Section under construction

Most of the website mentioned above for experiment building use a mix of GUI-based and code-based interfaces which allow for easy creation of most tasks. It’s not uncommon for the skeleton of an experiment to be built in a GUI for ease of translation into use in a browser, while supporting operations that cannot be performed in the GUI are executed in supporting code, depending on the platform. As more lab members create experiments on these sites, we hope this section can be populated with helpful tips on what works well and what does not.

 

Here is a document that outlines some basics of Pavlovia including porting an experiment from PsychoPy to Pavlovia. (Note: porting PsychoPy code has been challenging for some; it’s recommended to build experiments as much as possible within the GUI platforms rather than porting code.)

 

If you know JavaScript, however, you can program directly for the browser using PsychoJS.

​

​

Obtaining Consent

For all online studies, the consent forms are called ‘information sheets’ by the IRB. These are forms that the participants will view and accept rather than sign but they serve the same purpose and contain similar information as a consent form.

​

The information sheets will be presented to participants on the experiment hosting website (Gorilla, Pavlovia, or MTurk). You can show participants the PDF or you can copy / paste the important information into a text box on these sites. The participants can click an ‘Accept’ button in lieu of signing the form.

We have three information sheets (one for Prolific, one for audio-recording, and one for mTurk), which are linked to in the Forms and Flyers section of the Wiki under the Behavioral protocol documents section (in the online studies subsection).

​

Note: The information sheets say that participants will be paid within a particular range, for example $6.50 - 15.00. This may be confusing to participants if you present the PDF, so please be sure to specify the exact pay rate in the recruitment material.

​

​

Collecting Demographics

If you have online experiment participants fill out demographics forms on Google forms, they should be asked to put in a ‘participant ID’. They should be instructed to use either their Prolific ID or Worker ID for this, depending on the recruitment platform. This will allow you to connect their Google form demographic data to their experimental data as well as allow you to delete demographic data of participants whose experiment data is rejected. (Note that researchers using an auto-populating demographics Google form in Gorilla do not need to instruct participants to do this, as their information is automatically filled in. See Forms & Flyers for more details.)

​

Alternatively, you can administer the demographics form as a feature embedded in your task (e.g., a questionnaire embedded in your Gorilla task). This is recommended. The advantage of this is that the data are always linked to the demographics, rather than the demographics being administered one way (e.g., Google forms) and the data being elsewhere (e.g., Gorilla).

​

​

Lab Wide Accounts

We currently have shared lab accounts for Prolific, Gorilla, and Pavlovia. Passwords for these accounts are available on the Internal section of the Aly Lab website. Reach out to the lab manager for other questions about using these platforms.