Monday practical, part 1

Simple key- and button-press responses

The plan

The plan is to look at a simple grammaticality judgment experiment (that’s part 1) and then in part 2 to look at a slightly more complicated self-paced reading experiment. Remember, the idea is that you do as much of this as you can on your own (might be none of it, might be all of it) and then come to the practical sessions or use slack to get help with stuff you need help with.

A grammaticality judgment experiment

After working through the preparatory materials you should know just enough to make sense of a very simple grammaticality judgment experiment. I’d like you to download and run the code I provide, look at how the code works, and then attempt the exercises below, which involve editing the code in simple ways.

You need two files for this experiment, which you can download through the following two links:

The code makes some assumptions about where you will save it - you can change that if you want but then you’ll have to edit the code so it is looking in the right place, so it might be simpler to copy my directory structure. Create a new folder called grammaticality_judgments: if you are running the code on the server that needs to be in the public_html directory, if you are running the code on your local computer then it can be anywhere. Put three things in that folder: the jsPsych code you downloaded (for me that folder is called jspsych-6.2.0) plus the grammaticality_judgments.html and grammaticality_judgments.js files. So if you do that all correctly you will have a folder called grammaticality_judgments that cotains two files and a folder. On my mac it looks like this.

suggested directory structure

The idea is that for every experiment we will create a new folder that looks like this, and contains some code I have given you plus a copy of the jsPsych code.

(An aside: you might be wondering why we are including a separate copy of the jsPsych code with every experiment, rather than just having one shared copy that they all use. That’s a great question, and I originally used a single shared copy. But Josh de Leeuw pointed out that it’s better if every experiment is its own self-contained bundle that includes all the code it needs to run - otherwise, if we use a shared copy of jsPsych, when we update that shared copy we might break experiments that rely on features of an old version that no longer work. Including the jsPsych code separately with every experiment looks a bit redundant, but that redundancy provides robustness).

Assuming you have the directory structure all right, this code should run on your local computer (just open the grammaticality_judgments.html file in your browser) or you can upload it to the teaching server, put the grammaticality_judgments folder inside the public_html folder, and play with it there - assuming you have the directory structure as I describe, your URL will be http://jspsychks.ppls.ed.ac.uk/~USER/grammaticality_judgments/grammaticality_judgments.html, where USER is jspsych….

First, get the code and run through it so you can check it runs, and you can see what it does. Then take a look at the HTML and js files in your code editor (e.g. Atom).

You will see that grammaticality_judgments.html doesn’t have much in it - all that does is use the <script>...</script> tag to load a couple of plugins plus the file grammaticality_judgments.js.

The bulk of the code is in grammaticality_judgments.js. You will see that that code includes comments - these are bits of text that are enclosed in special markers so that the machine running the code ignores them. They are intended for you to read, to explain what the code is doing. But I’ll also add some explanation here.

grammaticality_judgments.js is probably one of the simplest types of experiments you could build. It has 4 grammaticality judgment trials, where participants provide a keypress response: y or n for “yes, this sentence could be spoken by a native speaker of English” or “no, it could not”. Note that is slightly different from what Sprouse (2011) does - he asks people for a numerical response rather than a simple yes-no, we’ll come to that later.

There is also a little bit of wrapper around those 4 trials - a consent screen where participants click a button to give consent and proceed to the experiment, some information screens before the experiment proper starts, and then a final screen where you can display debrief information, completion codes etc to participants.

The code starts by laying out the grammaticality judgment trials (NB just because they come first in the javascript file doesn’t mean they will be the first thing the participant sees - the timeline controls what participants see when). Each judgment trial involves showing the participant a sentence and getting a single keypress response from them, which we can achieve using the html-keyboard-response plugin. Details of the options for that plugin are in the jsPsych documentation. We are using the stimulus parameter to hold the sentence the participant is judging, prompt reminds the participant what they are supposed to be doing, and choices shows the list of keyboard responses they are allowed to provide - in this case we only accept y or n keypresses, so everything else is ignored. So the code for one judgment trial looks like this:

var judgment_trial_1 = {
    type: 'html-keyboard-response',
    stimulus: "Where did Blake buy the hat?",
    prompt: "<p><em>Could this sentence be spoken by a native speaker of English? Press y or n</em></p>",
    choices: ['y','n']
}

The only slightly fancy thing in there is that in the prompt I am using some HTML tags - the <p>...</p> tag to put the prompt in its own paragraph (it separates the prompt from the stimulus a bit, which I think looks better) and also the <em>...</em> tag to make the prompt in italics (again, to visually separate it from the stimulus sentence).

The code defines 4 such trials, inspired by the type of sentences used in Sprouse (2011).

That’s basically the only interesting part of the code! But we also need some preamble for the participants. Most experiments start with a consent screen, where participants read study information and then consent to participate. I include a placeholder for this consent screen using the button-response plugin - you see the consent information and then click a button to indicate that you consent. The code for that looks as follows:

var consent_screen = {
  type: 'html-button-response',
  stimulus: "<h3>Welcome to the experiment</h3>\
  <p style='text-align:left'>Experiments begin with an information sheet that explains to the participant\
  what they will be doing, how their data will be used, and how they will be\
  remunerated.</p>\
  <p style='text-align:left'>This is a placeholder for that information, which is normally reviewed\
  as part of the ethical review process.</p>",
  choices: ['Yes, I consent to participate'],
}

You will notice that the stimulus parameter here is quite complicated - it includes some HTML markup, including tags for headers (<h3> and </h3> to start and end a header), and paragraphs (<p> … </p>). By default, jsPsych centres all text, which sometimes looks fine (e.g. for the judgment trials, where we want the stimulus to be centred) but it looks horrendous for instruction text, so I also tell it to left-align that text, by adding some stuff inside the paragraph tags - I start a left-aligned paragraph with <p style='text-align:left'>, then end it with </p> as usual. Finally, the choices parameter for this trial type is a list of button labels - lists are enclosed in square brackets, here the list contains exactly one option with the “yes I consent” text, which produces a screen with exactly one button to be clicked. Finally, I have to use a backslash (\) character whenever I want to include a line break in the stimulus string, otherwise javascript thinks there is a syntax error.

I also define some information screens - these are html-keyboard-response trials, just like the judgment trials, but since I haven’t specified a restricted range of valid responses in the choices parameter, any key press will move the participant to the next trial. So for example the first instruction screen looks like this:

var instruction_screen_1 = {
  type: 'html-keyboard-response',
  stimulus: "<h3>Instructions</h3> \
  <p style='text-align:left'>In this experiment you will read English sentences, and determine if they sound\
  grammatical to you. By grammatical, we mean whether you think a native speaker of\
  English could say this sentence in a conversation. In other words, do you think it\
  would sound odd for your friends to say this to you, as if they don't speak English natively?</p>\
  <p style='text-align:left'>We are <b>not</b> concerned with whether the sentence would be graded highly\
  by a writing teacher: we do not care about points of style or clarity, and we do\
  not care about the grammar rules that you learned in school (who versus whom,\
  ending a sentence with a preposition, etc). Instead, we are interested in whether\
  these sentences could be said by a native speaker of English in normal daily speech.</p>\
  <p>Press any key for more instructions.</p>"
}

That’s quite a lot of text, but it’s just a very simple keyboard response trial with a long bit of HTML-formatted text to display.

Once all the various trials are defined, we can stick them together in a timeline for the experiment. The timeline is very simple and is just a list of all the trials we have created up to this point:

var full_timeline = [consent_screen,instruction_screen_1,instruction_screen_2,
                    judgment_trial_1,judgment_trial_2,judgment_trial_3,judgment_trial_4,
                    final_screen]

Then to run the experiment we call jsPsych.init with this full_timeline variable we have created. We also tell it to show the raw data on-screen at the end of the experiment (using the on_finish parameter of jsPsych.init). Obviously in a real experiment you would save the data rather than just showing it back to the participant, we’ll show you how to do that later in the course!

jsPsych.init({
    timeline: full_timeline,
    on_finish: function(){jsPsych.data.displayData('csv')}
})

Exercises with the grammaticality judgment experiment code

Attempt these problems.

Optional: a version of the code using timeline variables

You might have noticed that in grammaticality_judgments.js we quite laboriously lay out 4 judgment trials, all of which are identical in structure apart from the stimulus parameter. There are a couple of more efficient ways to do this, one of which is by using jsPsych timeline variables. If you’d like to see how that’s done, download and inspect the file grammaticality_judgments_with_timeline_variables.js, stick it in your grammaticality_judgments folder, then see if you can get that to run by telling grammaticality_judgments.html to load the timeline javascript file rather than the basic one.

You might be wondering what the advantage of using fancier code is, and/or thinking “I could just copy and paste the judgment trials and edit them directly, isn’t that simpler?”. It maybe is conceptually simpler to copy and paste simple code, but it’s also more error prone, since it relies on you not making any mistakes in copying, pasting and editing the same little block of code over and over. In general, if you find yourself doing a lot of copying, pasting and editing when writing code it’s a sign that you are doing manually something that the computer could do for you automatically, more quickly and with less chance of errors. We’ll come back to that in the next practical when we look at self-paced reading, where the “simple” manual approach would produce some really unwieldy code. I am not actually a massive fan of timeline variables - I find them a bit unintuitive - so we’ll be using other ways to build timelines, but the general point stands that constructing complex experiment timelines by copy-and-paste is error-prone and a bad idea.

Re-use

All aspects of this work are licensed under a Creative Commons Attribution 4.0 International License.

Course main page

Project maintained by kennysmithed Hosted on GitHub Pages — Theme by mattgraham