Netherlands Graduate School of Linguistics, January 2021
This is the webpage for the Netherlands Graduate School of Linguistics course Online Experiments for Language Scientists, running in January 2021.
Many areas in the language sciences rely on collecting data from human participants, from grammaticality judgments to behavioural responses (key presses, mouse clicks, spoken responses). While data collection traditionally takes place face-to-face, recent years have seen an explosion in the use of online data collection: participants take part remotely, providing responses through a survey tool or custom experimental software running in their web browser, with surveys or experiments often being advertised on crowdsourcing websites like Amazon Mechanical Turk (MTurk) or Prolific. Online methods potentially allow rapid and low-effort collection of large samples, and are particularly useful in situations where face-to-face data collection is not possible (e.g. during a pandemic); however, building and running these experiments poses challenges that differ from lab-based methods.
This course will provide a rapid tour of online experimental methods in the language sciences, covering a range of paradigms, from survey-like responses (e.g. as required for grammaticality judgments) through more standard psycholinguistic methods (button presses, mouse clicks) up to more ambitious and challenging techniques (e.g. voice recording, real-time interaction through text and/or streaming audio, iterated learning). Each day we will read a paper detailing a study using online methods, and look at code (written in javascript using jsPsych) to implement a similar experiment – the examples will skew towards the topics I am interested in (language learning, communication, language evolution), but we’ll cover more standard paradigms too (grammaticality judgments, self-paced reading) and the techniques are fairly general anyway. We’ll also look at the main platforms for reaching paid participants, e.g. MTurk and Prolific, and discuss some of the challenges around data quality and the ethics of running on those platforms.
No prior experience in coding is assumed, but you have to be prepared to dive in and try things out!
Kenny Smith (that’s me) is the ‘lecturer’ (I won’t be lecturing). Best way to get in touch with me is in one of the live sessions, via Slack (I’ll send details of the course Slack group), or by email to kenny.smith@ed.ac.uk.
We will meet 11:15 - 13:15 CET, Monday 18th-Friday 22nd January, on Zoom.
Course content will appear here - I’ll get everything up as early as possible to allow you to work through the materials before the course starts.
For each day I will provide a set reading and programming assignment(s). The reading involves a blog post introducing a published paper, you read both the blog and the paper. The programming assignments involve working through a section of the Online Experiments with jsPsych tutorial and/or looking at code implementing a language-related experiment. We can discuss this in the first class, but my plan is that we will use part of the Zoom sessions for a short discussion of the set readings, focussing on any bits you didn’t understand or wanted clarification on, but use the majority of the time to work on the practical assignments - I can help you work through problems you have running the code, difficulties you encounter in editing the code, etc. The idea is that this course will set you up to build and run your own experiments, so I am keen that we focus on the practical content.
I’d encourage you to work through as much of the material as you can before the course starts - if you can attempt some of the practical exercises (certainly the Monday practical) before we start it’ll mean you can identify problems and get my help on them in the limited time we have available. But at a minimum, work through the following preparatory materials.
Slightly ambitiously the plan is to cram a couple of basic experimental paradigms into the first day - hopefully the paradigms and the code are simple enough that we can manage this, but remember you have all week to make use of me in drop-in labs so if you don’t get through this on the Monday you can get help with it later in the week.
Pick one of the two topics and do the associated reading / practical. And if we don’t get this far and need to use Friday for catching up on earlier practicals, that’s completely fine - we are squeezing a lot in, you can always come back and look at this stuff later if we don’t get to it during the week.
Participant-to-participant interaction:
Iterated Learning:
All aspects of this work are licensed under a Creative Commons Attribution 4.0 International License.