Academic year 2021-2022
Be aware, this is the 2021-2022 version of this course
This edition of the course uses jsPsych 6.3. The 2022-2023 version of the course is now available and complete, which uses jsPsych 7.3. I recommend you use that! I am keeping this version up for completeness.
This is the webpage for the Honours/MSc Guided Research course Online Experiments for Language Scientists, running in academic year 2021/2022. I will add links to materials (readings, code) to this page; you will need to use Learn for electronic submission of your assessed work, and to keep an eye on announcements.
Many areas in the language sciences rely on collecting data from human participants, from grammaticality judgments to behavioural responses (key presses, mouse clicks, spoken responses). While data collection traditionally takes place face-to-face, recent years have seen an explosion in the use of online data collection: participants take part remotely, providing responses through a survey tool or custom experimental software running in their web browser, with surveys or experiments often being advertised on crowdsourcing websites like Amazon Mechanical Turk (MTurk) or Prolific. Online methods potentially allow rapid and low-effort collection of large samples, and are particularly useful in situations where face-to-face data collection is not possible (e.g. during a pandemic); however, building and running these experiments poses challenges that differ from lab-based methods.
This course will provide a rapid tour of online experimental methods in the language sciences, covering a range of paradigms, from survey-like responses (e.g. as required for grammaticality judgments) through more standard psycholinguistic methods (button presses, mouse clicks) up to more ambitious and challenging techniques (e.g. voice recording, real-time interaction through text and/or streaming audio, iterated learning). Each week we will read a paper detailing a study using online methods, and look at code (written in javascript using jsPsych) to implement a similar experiment - the examples will skew towards the topics I am interested in (language learning, communication, language evolution), but we’ll cover more standard paradigms too (grammaticality judgments, self-paced reading) and the techniques are fairly general anyway. We’ll also look at the main platforms for reaching paid participants, e.g. MTurk and Prolific, and discuss some of the challenges around data quality and the ethics of running on those platforms.
No prior experience in coding is assumed, but you have to be prepared to dive in and try things out; the assessment will involve elements of both literature review and coding.
The course is co-taught by Kenny Smith and Alisdair Tullo. Kenny (that’s me) is the main lecturer and the course organiser; Alisdair is the PPLS javascript/jsPsych guru and delivers the lab sessions with Kenny. Best way to get in touch with us is in one of the live sessions, see below, or by email to kenny.smith@ed.ac.uk or alisdair.tullo@ed.ac.uk.
We’ll also be supported in lab classes by three excellent tutors: Jess Brough, Aislinn Keogh, and Vilde Reksnes.
The course runs in semester 1. We have lectures 10am-11am on Mondays, and lab classes 9am-11am on Wednesdays.
Lectures and labs are both essential to doing well on the course - the assessment involves an understanding both of the literature on online experiments (covered in the readings and lectures) and the practicalities of how to build them (covered in your own work on the practicals, with support available in the labs).
Lectures take place in person on Monday mornings, 10am-10.55am, in 40 George Square Lecture Theatre B. I am very excited about this but also a bit nervous - you might be too - so please let’s do our best to look after each other. In particular:
Labs will take place online and we will use Gather (the one where you have a little avatar and can wander around a 2D world, bump into other people etc) - we have used gather for virtual labs in other courses, it’s great because it allows you to work alone or with friends, you can approach the tutor when you need help, talk live, share your screen to show your code, so it does pretty much everything you’d do in an in-person lab. Details of how to access Gather are on the course page on Learn.
There are two pieces of assessment, due on 11th November and 9th December. Assessment 1 is an annotated bibliography reviewing and evaluating 4 articles typically drawn from the course set readings. Assessment 2 is a more open-ended project where you produce a working experiment implemented in jsPsych and an accompanying report explaining the motivation behind that experiment, justifying important design decisions you took in building the experiment, and appraising the experiment and ways it could be improved/extended. Full details are provided in the assignment brief, and frequently asked questions will be answered in the FAQ.
Course content will appear here as we work through the course.
Each week there will be a set reading and a programming assignment. The reading involves a blog post introducing a published paper, you read both the blog and the paper, the lecture then provides an additional brief overview and an opportunity to ask questions/discuss the reading. The programming assignment involves working through a section of the Online Experiments with jsPsych tutorial and/or looking at (and editing) some code which implements a language-related experiment; you can use the lab classes as dedicated time to work on the programming task and get help with programming difficulties or questions you have.
In week 5 there is no lecture, but we have a lab as usual - use this time to catch up on labs from previous weeks, or to catch up on / get ahead with your reading.
No lecture or lab in week 11, but there are some materials that will be useful for you to read if you are thinking of setting up a real experiment in the wild!
These will be on Gather. Obviously we won’t write your code for you, but if you are having trouble interpreting an error message or finding a bug or want some tips on how to achieve a particular effect we can help you figure it out. Drop-in labs will be held at the following times:
All aspects of this work are licensed under a Creative Commons Attribution 4.0 International License.