Qualtrics Guide

Introduction

This guide walks you through the process of building a survey in Qualtrics for your research project. Qualtrics is a powerful survey platform widely used in academic research, and while it has a lot of features, you only need a handful of them to build a clean, functional experiment.

Unfortunately, it also has a lot of pitfalls. I have tried to include the most common (and most frustrating) issues as Warnings throughout this guide.

Work through this guide section by section as you build your survey — each section introduces the tools and concepts you'll need as they become relevant. You'll also find it useful to return to specific sections later, both during the survey creation process and when preparing your data for analysis.

This guide is still very much a work in progress. As you can imagine, it is difficult to put yourself back into the mindset of someone opening Qualtrics for the first time. If you have any thoughts or suggestions about this guide at any point, please share them in the Feedback Form at the end of the guide.

Your feedback at this stage would be invaluable and will help me continue to refine this guide for its intended audience. I have even set it up so that you can send comments anonymously. Personally, I sometimes hesitate to provide feedback on minor issues if I have to attach my name to it — so I'm providing this option for you in the hopes removing this potential barrier.

However, if you have immediate issues (with the guide or with Qualtrics), please email me directly: hillary.parent@yale.edu.

Also please note that this guide is still being updated, so if you leave it open in a tab, please make sure to refresh the page regularly so you'll see any changes.

Before you Begin

I know you might be tempted, but don't charge ahead and start creating your survey just yet.

You will save yourself both time and headaches down the road if you do things in a certain order, both because there are things to watch out for and because adding complexity (such as random assignment) to your survey too early will just slow you down. There are also a few things to keep in mind before you begin.

• Don't have your survey open in multiple tabs!
  I'm putting this warning up front because if you're like me, you'll be tempted to have different tabs open to edit different parts of your survey. Well... Don't do this!! Qualtrics even warns people about this — the way they have auto save set up may cause one tab to overwrite the changes you made in another tab, even across different areas / aspects of your survey.
• Be cautious when multiple people are editing simultaneously
  Qualtrics does not handle simultaneous editing well. If multiple collaborators are editing the survey at the same time, changes may conflict or behave unexpectedly. Try to coordinate with your team so only one person is editing at a time. (This likely stems from the above.)
• Preview your survey!
  You should be previewing your survey frequently — it is much easier to notice typos and wording issues when you are reading your items in a different context (i.e., while previewing it instead of in the Builder). You should (in my opinion) care how the survey both works and looks from a participant's point of view.

This is roughly the order you should create your survey in:

1. Start with your main question(s)
   Start with the wording and then explore the formatting options. Don't worry about the logic of your survey at this point (i.e., random assignment and conditions). There are multiple ways to set up conditions (which will be discussed in Section 7: How to Set Up Conditions), but at this stage you should focus on perfecting your items.
   Once everything looks right, you can start adding basic functionality, such as setting up two-part questions to appear dynamically, randomizing response options, etc.
   In short:
   1. Get the words right
   2. Make it look nice
   3. Make it work
2. Set up the pages of your survey
   Once your items have been perfected (the wording and how they are displayed), you can focus on setting up your survey properly — fully configuring each page of your survey. (This means finalizing your Questions (e.g., setting up Recode Values) and setting up the Blocks which contain them.)
3. Set up the logic of your survey
   Once your Blocks are set up, you can move on to the logic of your survey (random assignment, conditions, different branches, etc). You should test your survey extensively at this stage to make sure you have everything set up properly.
4. Add the Consent / Demographics / Screeners / Standard parts of your survey
   Having a lot of "bulk" in the way when you preview your survey will just slow you down, so don't do this until the main part of your survey is fully configured (and tested). The Survey Template will also have this done for you (although you may need to change the screeners, depending on your research question / design).

Below is a flow chart which shows the basic survey structure.

Flow Chart: The flow / logic of a simple survey.

Flow Chart: The flow / logic of a standard survey.

‹ › ⤢

I'm going to be using a running example to explain how to use Qualtrics. It may seem overly simple, but that's just to help simplify the explanations and make this guide easier to digest.

You are interested in asking people if they like one of three types of fruit:

With that in mind, let's begin!

Getting Started

Survey Templates

Survey Template - Styling Only — this file only has the custom CSS and various recommended Survey Options set up. Ideal for just starting out.

Survey Template - Survey Flow — this file has the custom CSS and Survey Options set up, as well as the basic Survey Flow elements (consent, screeners, etc.). Ideal when you're ready to put your survey together, after you're more familiar with Qualtrics.

** I'm probably going to be updating this one, since Shane wants you to use a simpler design.

Survey Template - Example Questions — this file has the custom CSS and Survey Options set up, and fully configured examples Questions. Ideal to help you see how all the options are configured for all the different Questions discussed in this guide. (Not ready yet, sorry!)

New Survey

Go to:

• yale.qualtrics.com︎ ⟵ SOM Qualtrics
  yalesurvey.qualtrics.com ⟵ University-wide Qualtrics (try this if the above doesn't work)
  
• Log in with your Yale NetID and password

Once logged in, you'll land on your Home page.

To create a new survey:

Create a new project (bottom left) → Survey → Get Started (bottom right)

From here, you can choose whether to start from scratch or import an existing survey.

A survey template has been provided for you to use as a starting point — it includes a custom design and some basic structure, so you don't have to start from scratch.

I have also included a survey which contains fully configured Questions, if you want to see how the options are set up as you read through this guide.

To import a QSF file:

How do you want to start your survey?   [Import a QSF file ▾] → Choose file → QSF File

Give your survey a meaningful name (e.g., Anchoring Effect Study — MGT 855 SP26) before clicking Create project.

The Interface

Here is a quick overview of what you'll see once you're inside a survey and the key menus you'll need to know.

Screenshot: Survey Builder Interface

⤢

First Steps

Survey Options

Survey Options ( in the Toolbar on the left) that you should set up now:

1. Under General (in the side panel), turn on New Survey Taking Experience
   
   🛑️Warning — New Survey Taking Experience

   You MUST opt into the New Survey Taking Experience or much of the information provided in this guide will be incorrect.

2. Under Security (in the side panel), turn on Prevent Multiple Submissions
   

   You can read more about this feature here.

Look and Feel

Look and Feel ( in the Toolbar on the left)

Below are my recommended settings. If you are using the Survey Template, you should verify that everything is set up as expected.

Screenshot: Look and Feel — Theme

Screenshot: Look and Feel — General

Screenshot: Look and Feel — Style

Screenshot: Look and Feel — Background

‹ › ⤢

Under Theme:

• Set the theme to Blank

Under General:

• (Optional): Add a progress bar
  • Progress Bar: With Verbose Text
  • Progress Bar Position: Top

Under Style:

• Primary Color: Don't change
• Display Text Width: Full
• Make Question & Answer text the same:
  • Question Text (px): 20
  • Answer Text (px):    20
• Multiple choice style: On (default)
• Custom CSS: Add my custom CSS (below)

Under Background:

• Background Color: Change if you want
• Question Container: On

Below is my custom CSS, it makes some simple improvements to the default Qualtrics CSS.
Press the edit button to open the CSS Editor.

Preview Button

Remember you should preview your survey frequently!

To share a Preview link, you need to set the permissions to Public:

Preview → Share preview (top-right)

Share Preview Menu:
Link visibility permissions: [Public ▾]

In the Share preview menu (below), click Copy link to grab the URL.

Publish Button

Use the Publish button (in the Builder Toolbar) as a manual Save throughout the building process — not just at launch. Don't worry, it sounds more official than it really is.

Qualtrics does auto save, but hitting Publish regularly ensures your changes are always committed.

There are cases where auto save fails — and I've been burned by this in the past (although it has gotten better over the years).

Technically, the proper way to manually save is to create a new version (below), but if you're lazy like me, you'll just use the Publish button. Doing this can make the values associated with the response options higher than expected, but if you follow my advice to set these values up yourself, this isn't an issue. It's not going to hurt anything either — it just affects how your Data File is generated, it doesn't change the data itself.

To create a new Version of your survey:

Builder Toolbar → Tools → Versions → Create new version

Building Your Survey

I'm going to advocate the frequent (but careful and deliberate) use of the Copy action for both Blocks and Questions.

Copy is particularly useful for highly customized surveys, especially if:

• You use the same Question Options (validation, requirements, JavaScript, etc.) frequently
• Entire pages of your survey have the same basic structure
• You have only minor changes between different conditions

Because most of your settings will remain intact! This means you will want to fully configure part of your survey (often a Question or an entire page/Block) and then simply make a Copy of it. Then you will only need to make minor adjustments (like changing the Question text and the Question name). Why waste time copy and pasting the same JavaScript into four different Questions when you could do it only once?

Of course, I'm not just suggesting this because it's faster and more efficient — it also ensures that you don't forget something. The caveat being that you use Copy deliberately, typically just after setting up a Question or Block, so what you need to change for the new Question/Block is top of mind.

Blocks

Blocks are the main unit of your survey — every Question will be placed in a Block. They are basically "containers" for your Questions. They are used to organize your survey behind the scenes.

Screenshot: Empty Block

My recommended setup is one main item (question / task / scale / two-part question / etc.) per Block. This way, a Block will represent one page of your survey.

This setup is the most robust — it keeps your survey modular, makes randomization work cleanly, and ensures that Response Time can be captured for individual items.

RT data is an extremely rich source of information, so don't throw it away — insert a Response Time Question into each Block!

Blocks will be referred to by their Names in the Survey Flow and in your Data File — so give them a meaningful name. Anchoring is much easier to work with than Block 4.

You can rename a Block by clicking on its name.

✅Best Practice

A properly configured Block will:

• Have a meaningful name
• Contain one task / scale / question
• Contain a Response Time Question (more on this in Section 3: Question Types)

And will NOT:

• Contain Page Breaks
• Have Question Randomization

Screenshot: A properly configured Block.

⤢

To add a new Block, click Add Block below any Block in the Builder. This will create a Block in that position.

Or you can use Block Actions (described below).

Block Actions

Block Actions are in the Ellipsis Menu (the ...) in the top-right corner of any Block.

Screenshot: Block Action Menu

Screenshot: Expanded Block Action Menu

‹ › ⤢

You can share a Block Preview Link the same way you do for your survey:

Block → ... → Preview block → Share preview → Link visibility permissions → Public → Save

In the Share preview menu, click Copy link to grab the URL.

Questions

The main part of your survey — the questions!

For each Question, try to follow roughly this order:

1. Set up the basics:
   • add question text and response option text
   • name the Question (and the Block)
   • set up the basic format options, like whether the options are Vertical or Horizontal
2. Set up the more complicated stuff:
   • add and test JavaScript (optional)
   • add and test Response Requirements (if any)
     • make sure it works properly
     • make sure you preview the Error Message on failure
   • add basic choice/statement randomization
3. Final touches:
   • set up Recode Values
   • add advanced randomization

📝Note

Qualtrics assigns an internal ID to every question (e.g., QID4). You can make these visible in the Builder by going to Tools → Show Internal IDs. Once enabled, the internal ID will appear next to the Question Name in the Builder. However, this is not a permanent setting, so you'll have to turn it on each time, as needed.

You may need this for two reasons:

1. to help understand your Data File
2. to write targeted CSS rulesets for specific questions (an alternative to JavaScript)

Naming your Questions does not cause you to lose the internal ID, it will always be preserved and can easily be seen by turning on the Show Internal IDs toggle.

To add a new Question, click + Add new question at the bottom-right of any Block.
A menu will appear containing all available question types — select the one you want and it will be added to the Block.

Alternatively, you can select the little + symbol at the top- or bottom-left corner of any question. These appear when your cursor is over a question and will create a default Multiple Choice question.

Screenshot: New Question menu expanded in the Builder

⤢

Question Actions and Menus

Just like Blocks, Question Actions can be found by clicking on the Ellipsis Menu (the ...) in the top-right corner of any Question.

There's also a right-click menu, which will display select Question Options for easier access, when you right-click on a Question outside the Question text area.

Screenshot: Question Action Menu

Screenshot: Expanded Question Action Menu

Screenshot: Expanded Question Right-Click Menu

‹ › ⤢

You can also select multiple Questions by checking the checkbox in the upper-left corner. When you do this, the Builder Panel will let you move the questions, Copy the questions, create a new Block and put the selected Questions in it, and even add Response Requirements.

Builder Panel when
multiple Questions are selected.

Selecting Multiple Questions.

Question Text

You can edit the text of the question easily in the Builder, just click in the Question Text and start typing.

If you want to format your Question Text, you can use the Rich Content Editor... (left) or HTML view (right).

The Rich Content Editor allows you to easily change the font (colour, size, family, etc.) and insert tables, images, links, etc.

The HTML View allows you do to the same, just with actual HTML. You can also view the exact HTML generated by Qualtrics in the HTML View.

Piped Text is something else entirely and I will save that until Section 7: How to Set Up Conditions.

Screenshot: Question Text Edit Menus

⤢

Response Option Text

A number of Question Types will have a "Choices" section in the Builder Panel that lets you set the number of response options.

For questions which collect multiple distinct responses from respondents for a single question, there will be a second option, typically called "Statements".

All of these work the same way and are part of setting up the text of a Question, regardless of whether it's the response options, scale points, labels, statements, etc.

Choice section of the Builder Panel.

Edit Multiple Menu.

With this you can easily adjust the number of choices. You can also explore suggested choices, which can be helpful if you want to use or build off of standard wording options.

The Edit Multiple menu is very useful — you can quickly and easily just type in the text of all your choices at once.

But like many things in Qualtrics, there are some issues. If you set certain Question Options (like Advanced Randomization and Recode Values), and then use the Edit Multiple menu, you may ran into some issues. One is a mere annoyance and the other much more problematic.

For Recode Values — it will revert your Recode Values to the default, which is not a huge deal but it is annoying.

For Advanced Randomization — it will break the randomization completely. And without any warning!

So, while I highly recommend the Edit Multiple menu, I do so with the caveat that you use it primarily when you are just starting your survey and stick with the (slower and more annoying but ultimately safer) method of individually clicking into each option to edit them after you've started to add Question Options just to be on the safe side.

🤔Consider — Labeling Each Choice

Consider whether you should label each choice for Likert scales. Often just labeling the endpoints is enough.

To do this, you should use the HTML entity   as the label for choices without text labels.   is called a non-breaking (nb) space (sp) and it will make sure the choice still appears, just with a blank label.

Below is a screenshot of how to set up the Edit Multiple menu with only labels for the endpoints of a 7-point scale.

For copy & paste purposes:

Copy Strongly<br>disagree &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; Strongly<br>agree

Question Types

Most experiments only need a few question types to collect all the data you need. This section covers the ones you'll use most, along with the key settings for each. The base survey includes an example of each of the following question types for reference.

You can change the Question Type for a Question in the Builder Panel — it will always be the first option.

Text / Graphic

Text / Graphic questions are the type to use when you want to provide instructions to your respondents without asking for or collecting a response.

The most common uses for this Question Type are:

• Landing pages
• Instructions prior to a task
• Instructions / statements that don't vary across conditions
• Debrief page

Multiple Choice

Multiple Choice questions are the most frequently used Question Type. Use them for any question where respondents select from a defined set of options.

Below are the Key Settings for Multiple Choice questions, which can be found in the Builder Panel.

Key Multiple Choice Question Settings

• Answer type
  • Allow one answer ⟵ most common
  • Allow multiple answers
• Format
  • Choose the type:
    • List ⟵ most common
    • Dropdown — (rare but can be useful)
    • Select box
  • Alignment (for List type):
    • Vertical
    • Horizontal
    • Column

🤔Consider — Whether Choices Should be Horizontal or Vertical

Builder Panel → Format → Alignment → Vertical / Horizontal

Likert scales should typically be presented horizontally, while common response options or a short list of options (like Yes / No) should (usually) be vertical.

Spend some time considering which orientation is more natural for your question and whether there may be order effects.

Matrix Table

Matrix Table questions display multiple questions with the same set of response options together in one table. They're the go-to format for multi-item Likert scales and any series of statements that share the same response options.

This is basically like having multiple Multiple Choice Questions within the same Question.

Although there are different types of Matrix Tables and many different options, I strongly suggest sticking with the default:

Matrix type:
Answer type:
Format:
Mobile friendly:
Everything else: Likert
Allow one answer
Standard likert
On
Off

If you deviate from these settings, the JavaScript and CSS offered in this guide probably won't work — I have not tested it.

The CSS I created preserves the whitespace in your Scale Point labels, meaning that it won't force your words to wrap if there's not enough space. I did this to remove the ambiguity: no more worrying about whether Qualtrics is going to force some of your labels onto two lines and not others — you will have complete control over this. Labels will only be on two lines if you add a (br)eak element (<br>) in between words like this: Strongly<br>agree. It's slightly more effort but worth it for the increased control.

Your questions (called statements in the Builder) will appear as rows, while the response options (called scale points in the Builder) will be the columns.

Text Entry

Text Entry Questions allow respondents to type out a response.

For Text Entry questions, there are a few things you should know about the custom CSS.

I changed the side of the textbox that the caret is in to be on the right (the default is left). If you want to change this, you'll have to change this in the custom CSS:

I also set it up so that Single Line Text Entry Questions have text boxes that are fairly small (only 55px), which is suitable for numeric responses.

For Multiple Lines Text Entry Questions, I made the text boxes a little larger, so they are suitable for short text responses. These are also the kind of textboxes that respondents can drag to make bigger.

This way you can utilize two different styles of Text Entry Questions without much fuss:

1. Use Single Line for numeric responses (ideal for inline text boxes)
2. Use Multiple Lines for short text-based responses

However, note that Essay Textbox type will have the same styling as the Multiple Lines type. This is because of how Qualtrics has implemented it. But as you can see in the Feedback and Comments section of the Survey Template, I have used JavaScript to adjust the size to override this custom CSS.

✅Best Practice

Consider writing your question so that the textbox is embedded within the question text itself (e.g., "I would estimate the value is ____"), rather than as a separate field below the question. This feels more natural for respondents and can improve data quality, though it does require a small amount of additional setup. See Section 5: JavaScript - Insert Text for details.

Screenshot: Default vs inline Text Entry Question

⤢

Form Fields

Form Fields allow you to collect multiple text entries within a single question, with each entry in its own labeled box. They are particularly useful for questions that require multiple numeric responses.

🛑Warning

Qualtrics changed this Question Type in their New Survey Design (one of the few downsides of the New Design), so the label is above the textbox instead of in front and there's no built-in way to override this. It's very easy to fix with CSS though — see Section 4: CSS - Form Fields for specifics.

You will need to adjust Form Field Questions individually, but you only need to change a single number: the width of the label. This will line the boxes up vertically.

If you're really, really against using CSS or JavaScript, you can stick some non-breaking spaces ( ) in the label to get them to line up.

Timing (Response Time)

The Timing question type records how long a respondent spends on a page. You should add one to every Block in your survey — it's one of the most useful pieces of data you'll collect, and it costs nothing to include.

To add one, select Timing from the question type dropdown. It won't be visible to respondents — it runs silently in the background.

When you export your data, Qualtrics will generate several columns for each timing question. The most important is {Question}_Page Submit, which records the total time from page load to submission. The other columns are based on mouse clicks and may not always record data depending on how the respondent interacts with the page, so they are generally less reliable.

Metadata

The Metadata Question Type records some basic information from your respondents, such as their browser, screen size, etc. I always include a Question of this type in the demographics section just in case.

Custom CSS

Toolbar → Look and Feel → Style → Custom CSS → edit (to expand the CSS Editor)

CSS controls the visual appearance of your survey, including fonts, spacing, button styles, etc.

When it comes to changing the appearance of Questions, anything you can do with CSS can also be done with JavaScript. The choice will depend on your personal preference, comfort level, and which you find easier to implement for your specific survey — if you have many Questions that you want to apply the same CSS ruleset to, you might find it easier to use CSS than to put the same JavaScript into each one. It's totally up to you!

Below are the cases where additional CSS code is needed, beyond what already comes with the Survey Template.

• Matrix Table Questions
• Matrix Table Questions with less than 3 statements
• Matrix Table Questions with less than 3 scale points
• Form Field Questions

CSS Basics

This is not intended to be a tutorial in CSS. Rather, this section is just to introduce you to the terminology so you can understand this section.

Format

CSS rulesets are formatted like this:

Whitespace

Whitespace in the selector is important!

So, don't add or remove spaces unless you know what you are doing!

!important

If you find something isn't working, try adding !important to the value of the property, just before the semi-colon, like this:

color: #232323; → color: #232323 !important;

If there are multiple rulesets affecting the appearance of an element, the !important says that this value takes precedence.

How to Apply only to a Specific Question

In general, these CSS rulesets are intended to apply to every applicable element. So, if you define a ruleset for a specific Question Type, then it'll apply to every Question of that type.

However, you can also apply rulesets only to specific questions. You can do that with JavaScript (more on this in Section 5: JavaScript - Adding CSS) or by using a Question's internal Qualtrics ID and CSS.

Finding out the internal ID of a Question is super easy! In the Builder Toolbar:

Tools → Show internal IDs

Yep, that's it! Now you'll see the internal ID right there in the Builder. It will always be of the form QID##.

Then, if you want a specific CSS ruleset to only target that question, all you have to do is add #question- followed by the unique ID (in the screenshot above, it's QID18) to the beginning of the line, like so:

How to Apply to Multiple Specific Questions

You can also list multiple IDs together to apply the ruleset to multiple questions. Just put the internal IDs (each preceded by #question-) as a comma-separated list within the :is() selector, like so:

How to Apply to All Except Specific Question(s)

What if you want to apply the ruleset to all but a specific question or questions? You can do that, too! Just put the internal IDs as a comma-separated list within the :not() selector, like so:

Matrix Tables Rulesets

For whatever reason Qualtrics still hasn't figured out the proper way to style Matrix Table questions.

I strongly urge you to use my custom CSS if you use Matrix Table Questions.

Some things to note:

1. If your Matrix Table has less than 3 statements, you probably shoudn't shade every other row
   • If you only have Matrix Table Questions with less than 3 statements, just remove this code from the Look and Feel CSS:
     Copy .question.matrix.table .matrix-content > :where(.matrix-row:nth-child(odd)) :is(.matrix-statement-header, .statement-wrapper) { background: color-mix(in srgb, #eaeaea 50%, transparent); }
   • If you have a mixture, then you should make sure the above rule only applies to the Questions with 3+ statements, using the :is() or :not() selectors described earlier (or you can use JavaScript instead)
2. If your Matrix Table has less than 3 scale points (e.g., Yes / No), you probably shoudn't shade every other row on the mobile view (because Matrix Tables turn into MC Questions for mobile)
   • If you only have Matrix Table Questions with less than 3 scale points, just remove this code from the Look and Feel CSS:
     Copy .question.matrix.table .matrix-accordion .foldout-items .likert-input:nth-child(odd):not(:hover) { background: color-mix(in srgb, #eaeaea 50%, transparent); }
   • If you have a mixture, then you should make sure the above rule only applies to the Questions with 3+ scale points, using the :is() or :not() selectors described earlier (or you can use JavaScript instead)

Now, the most annoying part about Matrix Tables — they need individual adjusting. The exact values will depend on:

• The length of your statements
• The number of response options

This is the what I used for the Paranormal Beliefs scale in the Survey Template:

Basically, you want to adjust the max-width property to make the entire table bigger or smaller and the grid-template-columns to adjust the ratio of space the statements and radio buttons take up.

The grid-template-columns is:

1. What % of the width do you want the statements to take up?
2. What % of the width do you want the radio buttons to take up?

They decided, from some reason, to use this:

Which says:

1. The statements should take up between a minimum of 6.25rem and a maximum of 23% of the width of the table
2. The scale points should take up the rest of the width (that's what 1fr means)

23% for the statements is ridiculous, so it's the 23% that we want to change.

Just play around with the numbers until it looks nice. Some more examples are provided below.

Moral Reasoning Item with default styling

Moral Reasoning Item with custom styling, specifically:
grid-template-columns: minmax(6.25rem, 75%) 1fr
max-width: 820px

Paranormal Beliefs Scale with custom styling, specifically:
grid-template-columns: minmax(6.25rem, 55%) 1fr
max-width: 900px

‹ › ⤢

Form Fields

One of the few things about Qualtrics' New Design that I don't like is what they did to Form Field questions. However, it's easy enough to fix with CSS.

If you are using these for numeric responses, you can make the textboxes identical to the text entry ones. Just put this in the custom CSS in Look and Feel.

The last thing that needs fixing is just to adjust the width of the labels so the textboxes line up. You can either do this here, using the :is() selector or use JavaScript on the particular question (or by using spaces (   ) in the label text). Just fiddle with the width until it looks right.

This is what I used for the Ice Cream Question in the Survey Template:

Screenshot: Aligning the textboxes for Form Field Questions.

JavaScript

JavaScript can be used to style Questions using CSS or for more functional things, like auto-advancing the page, inserting new HTML elements (i.e., text), etc.

Although Qualtrics does provide a Guide it is, quite frankly, terrible. Even I get confused trying to read it. There are also things they don't tell you or warn you about, that I have spent a lot of energy and effort figuring out — so even if you know JavaScript, take advantage of the information I'm providing and don't try to do it on your own.

Onload, OnReady, OnUnload

Builder Panel → Question behavior → JavaScript (alll the way at the bottom)

If you already have JavaScript on the Question, you'll see a little in the top-right corner. You can also click on this to open the JavaScript Editor.

The first thing to note is what you see when you open the JavaScript Editor for a Question:

Screenshot: JavaScript Editor

⤢

You'll see 3 functions already inside:

1. addOnload
2. addOnReady
3. addUnload

Each of these functions executes its code (what's written between the curly braces {}) at specific points in time. So, it's imperative that you put your code in the right one, otherwise it won't execute when it's supposed to or it may not even work at all.

Pay attention to these functions — you should not have multiple lines that say:

Qualtrics.SurveyEngine.addOnload(function()

Instead, put all code that should be in the addOnload function within the curly braces {} for that function.

It's fine to delete any function you aren't using or you can leave them — whatever you prefer. I always leave them in case I want to add more code later. But don't fret if you accidentally delete one.

There is also the addOnPageSubmit function, which is not included by default.

Add Text Before/After Text Boxes

This section will provide the JavaScript for inserting text around textboxes. Just make sure you use the correct code for your Question Type.

Inclined individuals will note that all we're doing is creating HTML objects and inserting them into the page with JavaScript.

📝Note

In the Builder, you will not be able to see the text you inserted with JavaScript. You may find this annoying and it may also confuse your collaborators.

This is one reason why naming your Block and your Question is so important — because if the entire question text is inserted with JavaScript, it will look blank in the Builder.

However, you can easily see the text by previewing the Question or the Block. See the screenshots below.

View of the Builder for a Text Entry Question with text inserted with JavaScript

Preview of a Text Entry Question with text inserted with JavaScript

‹ › ⤢

✅Best Practice

In your Data File the Question Text will be blank unless you add the text to your Question (like normal) and surround it with a div set to not be displayed in the HTML View, like so:

<div style="display:none;">0.875 is the same as __ / 8</div>

This is really easy to do and I highly recommend it — this way you'll have a record of the question text in your Data File.

HTML View of a Text Entry Question, with Question Text in a div which won't be displayed to participants.

⤢

Text Entry Questions

Add this JavaScript to the addOnload function of each question you want to add text to.

Note that this assumes you have the custom CSS in the above section. If you don't, see the next code block.

You only need to change (at most) the first two lines (that start with var).

For the example above, this is what I used (note the space at the end of beforeText and the beginning of afterText):

If you don't have the accompanying CSS (specifically, the .beforeSpan and .afterSpan rulesets) use this:

Form Field Questions

Preview of a Form Field Question with text inserted with JavaScript

Add this JavaScript to the addOnload function of each question you want to add text to.

Since Form Fields already have a label, this is only if you want text after the textbox.

This will assume you want the same text after each textbox in a Form Field question. This seems like the most likely use case, but I may add a more general version later/on request.

Note that this assumes you have the CSS ruleset in the above section. If you don't, see the next code block.

You only need to change the var afterText = ""; line.

If you don't have the accompanying CSS (specifically, .afterSpan ruleset) use this:

Auto-Focus the Text Box

Add this JavaScript to the addOnReady function if you want the textbox to be auto-focused, so the respondent can start typing right away.

For a more generic function that will work on the first textbox element on the page (in case you have multiple questions and the items are randomized), add this to the addOnReady function of one question (not all of them!).

Auto-Submit

Use this code to auto-submit a page once a respondent has selected a response for each question.

One Question per Page

This code is for a Multiple Choice / Matrix Table Question with either Radio Buttons or Checkboxes, (i.e., Single or Multiple Answer formats).

Note that here, there is code in addOnload and addOnUnload and top-level code outside these functions. For that reason, I have wrapped the entire block in an IIFE (immediately invoked function expression). So, you need to copy & paste the entire block of code, overwriting the default function definitions in the JavaScript Editor. Note that you can add additional code to the functions like normal after you do this — the important part is having the whole block wrapped in (function() { })();.

Multiple Questions per Page

Only put this code on ONE question on the page, not all of them!

Also make sure that the page only has Multiple Choice, Matrix Table, and/or Text / Graphic Questions on it.

If you have other Question Types (like Text Entry), this code will auto-submit the page when they have answered all the Multiple Choice & Matrix Table Questions, even if they haven't answered the Text Entry Question yet.

Adding CSS

You can either do this with CSS in the custom CSS section of Look and Feel OR you can use JavaScript to apply a CSS ruleset to a single question.

Recall that these are the cases where additional CSS code is needed, beyond what already comes with the Survey Template:

• Matrix Table Questions
• Matrix Table Questions with less than 3 statements
• Matrix Table Questions with less than 3 scale points
• Form Field Questions

Basics

If you know how to write CSS or have a CSS ruleset already written, then adding custom CSS to a Question with JavaScript is a cinch.

Say you have the following CSS ruleset:

(Recall that this is what we used to adjust the Paranormal Beliefs scale in the CSS section.)

Then, all we have to do is use JavaScript to create an HTML style element, put the CSS ruleset inside it, and append it to the page.

Simply put this in the addOnload function:

For a more general form:

Then just change [REST OF SELECTORS] and add as many property: value; lines as needed / desired.

Hopefully this helps explain why I said you can either use CSS or JavaScript — because it's basically the same thing!

Matrix Tables

Add this to addOnload for each Matrix Table Question and fiddle with 900 and the 55% in the code below until it looks right:

Recall that you can adjust the width of the table itself (the max-width property) and that the grid-template-columns property means:

1. The % of the width you want the statements to take up
2. The % of the width you want the radio buttons to take up

Form Fields

All you have to do for Form Fields is adjust the label width, making it the same so the textboxes line up.

Add this to addOnload and fiddle with the 120 below until the text boxes line up:

Screenshot: Aligning the textboxes for Form Field Questions.

Two-Part Questions

In my opinion, the best way to do a two-part question where the response to the previous question is "locked" in, is to simulate Display Logic.

In other words, we're going to create two separate (but identical) Questions. On the first page, they only see the first Question. And on the second page, they see both Questions.

Then we can Carry Forward their response and add JavaScript to disable the Question, so they can't change it.

This makes sure:

• We don't somehow corrupt their previous answer
• We don't lose the RT data for the first response
• We have separate RT data for the second response

Text Entry Questions

This is easiest to do for Text Entry Questions.

1. Fully configure the Block for the first question
2. Copy the Block
   • For the repeated first question, add their response to the real first question as a Default Choice:
     Builder Panel ⟶ Question Behavior ⟶ Default choices ⟶ Survey Question ⟶ (select the question) ⟶ (select the option which shows the text of the question)
   • Add the following JavaScript to the onLoad function:
     Copy var input = this.getQuestionContainer().querySelector(".text-input"); input.disabled = true; input.style.webkitUserSelect = "none"; input.style.msUserSelect = "none"; input.style.mozUserSelect = "none"; input.style.userSelect = "none"; // optional - make it look locked in input.style.color = "grey"; input.style.backgroundColor = "#e5e5e5";
   • Add your follow-up question below

Screenshot: View of the Builder for a Two-Part Text Entry Question

Screenshot: Default Choice Menu — Carry Forward Answer

‹ › ⤢

Multiple Choice Questions

(to do)

Question Options

The following settings are available on individual questions via the Builder Panel and can be applied to any Question Type where relevant.

Response Requirements

You can put restrictions or conditions on participants' responses.

There are two main options:

1. Add requirements
   1. Force response — respondents can't progress until they answer the question
   2. Request response — if participants try to progress without answering, they will be prompted to answer the questions or continue without answering
2. Add validation
   • used to ensure responses pass certain conditions
   • options will depend on Question Type

Response Requirement section
of the Builder Panel

Response requested prompt.

There are two common uses for validation:

1. Numeric responses for Text Entry Questions — ensure that participants only submit numerical answers
   Add validation: On → [Content type ▾] → [Number ▾]

   You can further restrict their responses by adding a Minimum / Maximum / Maximum decimals.

   Screenshot: Response requirements section of the Builder Panel
   How to set up numeric response validation for Text Entry Questions.

   Note: You must use Single Line Text Entry Questions for this option to appear.

   You can do this for Multile Lines / Essay Text Box type Text Entry Questions, but you'll have to use Custom Validation and "Matches Regex" (but you also have to know Regex).

   To restrict it to integers (i.e., no decimals), you would use:
   /^[0-9]+$/
   To accept numbers with commas and decimals (and numbers with the leading 0 omitted), use:
   /(^[0-9]+(,[0-9]+)*(\.[0-9]+)?$)|(^\.[0-9]+$)/
   That should mimic the default Qualtrics number validation, but you could further restrict it to accept only valid comma usage, etc.

   If you want to get really fancy (and you can use this instead of the built-in Qualtrics numeric response validation), you can use:

   /^((([1-9][0-9]{0,2}((,[0-9]{3})+)?)|0)(\.[0-9]+)?|\.[0-9]+)$/

   Which accepts properly formed non-negative numbers:

   ✅ 0, 5, 123, 9999, 987654321
   ✅ 1,000, 25,000, 123,456,789
   ✅ 0.5, 1.23, 1000.555, 1,000.555
   ✅ .5, .123

   ❌  , ., 0., 123., 4,5, 1,23
   ❌ 00, 01, 0123
   ❌ ..5, 1..2, .5.6
   ❌ -1, -0.5-.5

2. Comprehension checks — force participants to select the correct answer before they can continue
   Add validation: On →
   
   

   Custom Validation Menu

   [Question ▾]   [Correct Answer ▾]   [Is Selected ▾]
   Choose an error message: [My Library: Your Name ▾]   [New Message... ▾]
   
   

   New Message Menu

   Click on: + Create new variation →
   Resource name: Incorrect, try again
   Validation error: That's incorrect, please try again.
   Error message:   That's incorrect, please try again.

   This means participants will have to choose the selected choice to progress in the survey, which is useful if you want to make sure participants understand the task you will be asking them to do.

   You will also have to set up a custom error message, because there are no good built-in options for this. I suggest something like:
   That's incorrect, please try again.
   (which I wrote out above)

   

   Screenshot: Custom Validation Menu

   How to set up comprehension check failure for a Multiple Choice Question.
   In this case, participants won't be able to progress until they select "Yes" to the "Do you like apples?" question.

   

   Screenshot: New Message Menu

   How to create a custom error message.
   The "Validation error" is the text that will appear in the selection menu, while the "Error message" is what will display to participants. I recommend putting the full text in the "Validation error" textbox, that way you know what will display to participants without having to check.
   If you're still confused, you can read more about this here.

   ‹ › ⤢

Question Behavior

There are several options in the Question Behavior section of the Builder Panel.

Question Behavior Options

• Display logic — will display a question only if the specified conditions are met
  Common use: two-part or follow-up questions
• Skip logic — will skip entire sections of your survey if specified conditions are met
  I've never used this feauture, I prefer using Survey Flow for this.
• Carry forward choices — carries forward the options from a previous question
  Common use: carry forward randomized order of options
• Choice randomization — randomize the order of the choices
  Depending on the Question Type this may have a different name and there may even be two of these (i.e., for Matrix Tables)
• Recode values — the values that will be associated to the Questions and the Responses in your Data File
• Default choices — preselect an option or put default text into a textbox.
  Very uncommonly used.
• JavaScript — add JavaScript to your Question

For the latter four options above, along with Response requirements (forced / requested / validation), icons will appear in the upper right corner of the Question in the Builder:

This means you can quickly scan your Questions to make sure you have certain things set up.

Plus, for all except Response requirements (the * ), if you click on the icon, the associated menu will open — a super helpful shortcut!

The last two above both lead to the Recode Values menu: the left is Recode Values and the right is Variable Naming.

Display Logic

Display Logic is a question-level option that lets you show or hide a question based on certain conditions — for example, a previous response.

Two-Part / Follow-up Questions

A common use case is a two-part or follow-up question, where a second question should appear if the respondent gave a specific answer to a previous question. The second question should appear dynamically without requiring the respondent to advance to a new page.

For example:

The Question: "Have you ever been married?" should only appear if the respondent answered "No" to "Are you currently married?".

To set this up, add Display Logic to the follow-up question and check "In page" in the bottom-left, then set the condition to: Are you currently married? → No → Is Selected

In general:

However, if you want to lock in their response to the first question, you have to do a little more work using JavaScript, see Section 5: JavaScript - Two-Part Question for details.

Screenshot: Preview of a two-part question

Screenshot: How to set up a two-part question in the Builder

Screenshot: Display Logic menu for a follow-up question.

‹ › ⤢

Choice Randomization

Although I'm referring to this as "Choice Randomization" (because that's the most common), it might have a slightly different name depending on the Question Type. For example, it's called "Form field randomization" for Form Fields and "Scale point randomization" for Matrix Tables.

Some Questions, namely Matrix Tables, will also have a second randomization option: Statement randomization, which randomizes the Questions rather than the answers.

Either way the randomization works the same, just be careful which you randomize if there's more than one option in the Builder Panel.

The options for randomization are:

• No Randomization
• Display answers in a random order
• Present only __ of total choices
• Randomly flip the order of choices
  • Consistently reverse the choice order for all questions with this randomization option selected
    ↑ not recommended, will be checked by default
• Advanced Randomization — (brings up a separate menu)

The most common use cases will be:

• Full randomization
• Randomly flip the order (with the box unchecked)

Advanced Randomization

Advanced Randomization can be used if you want the order of some of the options to be fixed. For example, if you have 3 options and "All of the above". You would use Advanced Randomization to randomize the first 3 options and leave "All of the above" as the last option.

Just move the 3 choices you want to randomize into the "Randomize choices" list. See the screenshots below.

Screenshot: Multiple Choice Question — Advanced Randomization Menu

Screenshot: Multiple Choice Question — Advanced Randomization Setup for 3 Randomized Choices and 'All of the Above'

‹ › ⤢

⚠️Warning — Advanced Randomization

Always set up Advanced Randomization as your last step, after all option text is finalized. Editing option text using the Edit multiple button after setting up Advanced Randomization can silently break the randomization — Qualtrics will give no warning that anything is wrong. This is a known Qualtrics issue.

This is where extra caution is needed if you use the Edit multiple menu.

Below is what happens if you use the "Edit multiple" menu after setting up Advanced Randomization — everything was booted back to the "Fixed display order" section and you have some "phantom" options that you can't move. If this happens, just remove randomization altogether, and then set up Advanced Randomization again. That seems to remove the "phantom" options.

But if you don't check this, you won't know anything is wrong!

Screenshot: Broken Advanced Randomization

⤢

Recode Values

Qualtrics automatically sets the value of response options in your survey incrementally. This can make understanding your Data File difficult, especially if you "Publish" and then add more Questions/Response Options to your survey.

I highly recommend setting up Recode Values and Variable Naming (and Question Export Tags for Matrix Table Questions).

Doing this step now will save you time later. Plus, they transfer to Questions that you Copy — another reason you should fully configure one Question or Block and copy it for similar ones! (Although recall the warning about using the Edit multiple Menu and recode values resetting.)

Although the menu itself is called Recode Values, it will bring up a dialogue where you can set the Recode Values, the Variable Naming (and Question Export Tags for Matrix Table Questions).

Recode Values are the numerical values assigned to each response.

• If you have a bipolar scale, consider assigning:
  • negative numbers to one side
  • positive numbers to the other
  • and 0 to the middle or neutral option (if there is one)
• If you have a unipolar scale, consider assigning:
  • 0 to the option that implies the absence of the construct (e.g., "Not at all")
  • positive/negative numbers to the rest
• For binary choices (like Yes / No), consider assigning 1 to "Yes" and 0 to "No", instead of the default Yes (1), No (2).
  This is also useful for Multiple Choice questions where one answer is considered "correct" and the other answer "incorrect".

Variable Naming are the text values assigned to each response.

This can be important to set up if you have very long response options or your options have formatting. For example, if one of your options is: Option <b>A</b>, that is what will appear in your Data File.

Question Export Tags will be the column names in your Data File for each question in a Matrix Table.

When you download your Data File, you will either download the responses as "Values" (the Recode Values) or "Labels" (the full option text or the Variable Naming if you have that set up) — which is why this step is important. This will be explained in more detail in Section 10: Downloading Your Data.

Screenshot: Recode Value Menu for a Multiple Choice Question

Screenshot: Recode Value Menu for a Multiple Choice Question with Recommended Setup

Screenshot: Recode Value Menu for a Matrix Table Question

Screenshot: Recode Value Menu for a Matrix Table Question
After you check all the checkboxes, you can see that the first question is called Q73_9 and that will be its column name in your Data File, unless you change it.

Screenshot: Recode Value Menu for a Matrix Table Question with the Recommended Setup.

‹ › ⤢

Survey Flow

Toolbar → Survey Flow

The way your survey is organized behind the scenes is just as important as the questions themselves.

The Survey Flow is where you control the order and logic of your entire survey — which blocks are shown, in what order, to whom, and under what conditions. Think of it as the skeleton of your experiment.

Each Block you create in the Builder will automatically appear in the Survey Flow as an element.

Survey Flow IDs

Much like Questions, Survey Flow elements also have unique internal Qualtrics IDs. To view the internal IDs of Survey Flow Elements, click the "Show flow IDs" toggle in the top-right corner of the screen. Each ID will be in the form of FL_##. You may need these to understand your Data File. You will also have to toggle this on every time you view the Survey Flow, as it's not a permanent setting.

Survey Flow Elements

In the Survey Flow you can add new types of elements — including Randomizers, Embedded Data fields, and Branch Logic — by clicking + Add a New Element Here or Add below on any element.

Blocks

You can use this option to move or add Blocks that you created in the Builder in the flow of your survey.

The order of the Blocks in the Survey Flow will affect the order of the Blocks in the Builder.

If you delete Blocks from your Survey Flow, don't worry — they aren't really deleted, they just won't be shown to your respondents. They will move to the bottom of the Builder and have a "Not in Survey Flow" indicator:

Sometimes it's just easier to Add a Block in a specific location than dragging and dropping it. Just make sure you don't have duplicate Blocks showing to the same respondents unless you actually want to.

Embedded Data

Embedded Data fields store information about each respondent that isn't directly tied to a question — for example, which experimental condition they were assigned to. This data is stored behind the scenes and exported with your Data File. It can also be accessed using Piped Text or accessed, set, or even updated using JavaScript. However, Embedded Data elements specifically for use with JavaScript have a separate naming convention — they must start with __js_.

Randomizer

The Randomizer element is the engine behind most experimental designs.

But the key is in the number of elements that will be presented:

Screenshot: Randomizer set to present all elements for block order randomization

⤢

Qualtrics will automatically update the number of elements to present to be the total number of elements as you place elements in the Randomizer.

But you can adjust this number (let's call it N) or adjust the elements you place into the Randomizer for specific purposes. Typically, you will either set:

• N = total # of elements ⟶ Randomize the order, but still show respondents every element, or
• N = 1 & Evenly Present Elements ⟶ Randomly present one of the elements to respondents
  • No duplicate elements in the Randomizer (typical) ⟶ this is how you set up conditions
  • Duplicate elements in the Randomizer (rare) ⟶ this can be useful if you want one condition to have more respondents than another. For example, if you want participants assigned to your Experimental condition and your Control condition at a ratio of 2:1, you would put 2 Embedded Data elements for the Experimental condition in the Randomizer and only 1 for the Control condition. See the image below.
    Screenshot: Randomizer with a 2:1 ratio

    ⤢

⚠️Warning

Always manually click the + button on the Randomizer count until it stops incrementing. Qualtrics should update the count automatically as you add elements, but it's better to verify this yourself.

There are also cases when the count will be incorrect — for example, if you add a Block to your survey while a Randomizer already exists. See below:

Screenshot: Survey Flow showing a Randomizer with an incorrect element count.
I set up the Randomizer and went back into the Builder, where I added two Blocks below the "Screeners - Zip code" Block. The Blocks were automatically placed into the Randomizer but the count did not increase.

⤢

This is why it's better to finish creating all your Blocks and Questions before you set up anything in the Survey Flow.

⚠️Warning

Before you launch your survey, you must reset the Randomizer count for any Randomizer that you selected Evenly Present Elements for.

The "Edit count" menu will appear once you check the Evenly Present Elements option.

If you previewed your survey, the Randomizer will have already started counting those — resetting ensures the count starts fresh with real participants. This is easy to forget, so it is included in the Pre-Launch Checklist in Section 9: Pre-Launch Checklist.

Screenshot: Randomizer in Survey Flow with Evenly Present Elements checked.
Note the "Edit count" menu.

Screenshot: Edit count menu for a Randomizer.

‹ › ⤢

Branch Logic

Branch Logic lets you direct participants to different parts of the survey — or out of it entirely — based on their responses, Embedded Data, or various other conditions.

Set the condition, then nest the relevant elements inside the Branch by clicking Add a New Element Here within it or dragging and dropping an already present element.

The nested elements will be shown to participants when the condition is true, and skipped when it is false.

Screenshot: Branch Logic condition menu with dropdown expanded.

Screenshot: Branch Logic condition menu — Embedded Data

Branch Logic condition menu — Question

‹ › ⤢

Groups

Groups let you group sections of your survey together. This is useful if you have more complicated designs, such as randomizing the order of scales, each of which has their own items randomized.

The downside to Groups is that even though you can name them, their names are not (currently) exported in the Data File. Instead, you'll have to make note of their internal Flow ID to understand your Data File.

Or you can use my little trick, which will be explained in the next section.

End of Survey

The End of Survey element immediately sends participants out of the survey.

For this element, you will either want to redirect participants back to the platform (required for Prolific) or set a custom message to display (used with MTurk), depending on why they are being sent out of the survey.

Unfortunately, Qualtrics saves these custom messages to your account library, so they are not able to be transferred with a QSF file. This means you will have to set them up yourself. They are easy to set up though and saving them to your library will allow you to reuse them later. But again note that this can cause problems with imported surveys.

The three specific "End of Survey" routes will be explained in the next section.

Survey Structure

1. URL Parameters
2. Consent
3. If did not consent, send to End of Survey (did not consent)
4. Screeners (optional)
5. If failed screeners, send to End of Survey (screened out)
6. Landing page
7. Randomly assign to a condition
8. Main tasks
9. Exploratory / Additional / Full Sample Tasks (optional)
10. Demographics
11. Comments / Feedback
12. Debrief (optional)
13. End of Survey (completed)

The above is the basic structure of your survey, where these elements are elements in your Survey Flow and are not visible to participants.

One point to consider: it is considered good practice to still pay participants who fail screeners for their time, usually at a rate much lower than the base survey.

Prolific has built this into their platform, allowing you to create different "completion paths" for your survey, each with a different URL. You can set different compensation rates for these paths and set a limit on how many "screened out" participants you want to pay — if this limit is met, your survey is paused until you adjust this amount. Go here for more details.

The Template Survey assumes that you will be using Prolific and has set up the "End of Survey" elements to make this clear — however, you will have to get the real URLs once you have your experiment set up on Prolific.

If you don't use Prolific, you will have to set up two Qualtrics surveys — one for just the Consent and Screeners and one for everything else — with the first survey redirecting participants to the full survey if they pass all the screeners or displaying a message (like: "Thank you for your time, but you do not qualify to participate in this survey.") if not.

URL Parameters

You should create an Embedded Data element at the very beginning of your Survey Flow. The field names will depend on what platform you are using. These elements capture participants' platform IDs and help enable a smoother payment process.

Prolific (link for more info):

MTurk (link for more info):

Screenshot: How to set up URL Parameters in the Survey Flow

⤢

Other Embedded Data

I'm also going to recommend that you add the following built-in Qualtrics Embedded Data Fields:

I have not personally used them (yet), but I discovered them while writing this guide and I think they might be a good idea to include. You can read more about them here.

Consent

Before collecting any data — including screener responses — participants must consent to the study. This is a research ethics requirement. If a participant does not consent, they are redirected to the end of the survey immediately via Branch Logic.

You do not need to pay these participants for their time.

Simply add a Branch Logic element below your Consent Block and set the condition to:
[Consent Question] [No] [Is Selected]
and place an End of Survey element inside.

If you're using Prolific you should create a Block with a Text / Graphic Question with the following message:

And place it within the Branch Logic you just created, before the End of Survey element.

End of Survey element → Customize → Override Survey Options → Redirect to a URL...
https://app.prolific.com/submissions/complete?cc=REFUSED_CODE

Replace REFUSED_CODE in the URL with the code from Prolific after you set up the completion paths.

Screenshot: The Survey Flow logic for consent, with the Customize menu for the "did not consent" End of Survey element.

Replace REFUSED_CODE in the URL with the code from Prolific after you set up the completion paths.

⤢

For MTurk, you should set up a custom message within the End of Survey element. The suggested message is:

Screeners

The base survey screeners are designed to check for English fluency and confirm that participants are paying attention and putting in genuine effort. These are data quality checks — they help ensure that the responses you collect are meaningful. Participants who fail a screener are also redirected to the end of the survey via Branch Logic.

You can Duplicate the End of Survey element from the Consent step above — just make sure you update the code at the end of the URL to the correct one.

If you included the Q_TerminateFlag Embedded Data Field suggested earlier, you can also check the "Flag Response As" option (under Customize for the End of Survey Element) and set it to "Screened Out".

If you are redirecting participants (like you should with Prolific) you also need to create a Block with a Text / Graphic Question, informing participants that they did not qualify for the survey, but will still receive compensation. In the Survey Template, I have it set up as:

If you're using MTurk, you should use a custom end of survey message instead. How you word it will depend on how you're setting up your screeners, i.e., if you are paying those who do not qualify.

Screenshot: The Survey Flow logic for screeners, with the Customize menu for the screened out End of Survey element.

Replace SCREENED_OUT_CODE in the URL with the code from Prolific after you set up the completion paths.

⤢

Completed Survey

Make sure you add an End of Survey element at the very end of your Survey Flow. Set it up the same way you do the others, but make sure to put the correct code at the end of the URL if using Prolific:

If using the Survey Template, replace COMPLETION_PATH in the URL with the correct code from Prolific after you set up the completion paths.

Or you can Duplicate and adjust the URL, as before.

Although this goes at the end of your Survey Flow, I'm putting this here because you might as well set up all the End of Survey elements at once.

If you included the Q_TerminateFlag Embedded Data Field suggested earlier, you can also check the "Flag Response As" option (under Customize for the End of Survey Element) and set it to "Survey Complete".

Screenshot: The Customize menu for the final End of Survey element. Replace COMPLETED_CODE in the URL with the code from Prolific after you set up the completion paths.

⤢

For MTurk (through CloudResearch), there's a specific way to set up the end of survey message to generate a Secret Key for participants to copy & paste to receive their payment — which you can find the instructions for here.

Below I will paste the code you have to add to your custom End of Survey message:

This is slightly different than the code I have saved, which is:

There is also the random number generator option, which I used prior to CloudResearch being launched. You can read about how to set that up here.

Random Assignment

Although you might be tempted to just create a Randomizer and place your Blocks into it to take care of random assignment — I highly recommend against this approach.

Rather, you should create a Randomizer and place an Embedded Data element it in. Set the field name to be condition (or similar). Duplicate this element, so you have one for each condition. Then set the value of these elements so that each one represents one of your conditions. (You can Duplicate before or after setting the value — it depends how close your condition names are.)

For the Randomizer, set:

• number of elements to present = 1
• Evenly Present Elements

Screenshot: How to set up random assignment in the Survey Flow using Embedded Data.

⤢

This will allow you to refer to this value elsewhere, which is useful for more complicated surveys.

This will create a "duplicate" column in your Data File. You can remove the less sensible one in your working file. One column will be the Embedded Data column and the other will be the randomized order of the Randomizer (which you will get if you download the randomized order, see Section 10: Downloading Your Data for more details.

In the screenshot above, this would be the column FL_7_DO, since FL_7 is the Flow ID of the Randomizer and DO means Display Order. In this case, this column will contain the values: FL_8, FL_9, FL_10, which are the IDs for the Embedded Data elements for the conditions. You can verify that this column matches the one that was automatically generated for the condition Embedded Data element. Below is a screenshot of these columns in Excel.

Randomizing Block Order

If you are randomizing the order of some of your Questions, such as the order of items in a scale, in most cases it will be fine to just randomize the Blocks without creating an Embedded Data element to capture the order.

For this, you would:

1. create a Randomizer
2. place all the elements you wish you randomize in it
3. Set the number of elements to present = total # of elements

Although in the Data File, the column name will be the Flow ID of the Randomizer, the values in the column will make it clear what was randomized (well, as long as you named your Blocks!).

You will see a | separated list of Block names (typically with the spaces removed but sometimes truncated, as well), in the order participants saw them. For example: Fruit-Oranges|Fruit-Bananas|Fruit-Apples. See below.

Screenshot: How to randomize the order of items in the Survey Flow.

⤢

If you don't want to fully randomize the order of items, but have specific orders that you want to randomly assign participants to see, you would create a condition Embedded Data element like above and use Branch Logic, which will be described in the next section — Section 7: Survey Flow - Order Conditions.

How to Set Up Conditions

This section explains how to set up conditions that show different content to participants, based on their condition.

There are a few options for how to show participants the right content.

Each option has pros and cons, choose based on the complexity of your design and your comfort level.

Branch Logic

This is likely the one you will opt to use, as it's the most straightforward and intuitive.

The Branch Logic approach uses the Branch Logic element in Survey Flow to display the correct Blocks to participants.

In the Builder:

• Create a full Block for each of your conditions separately
  (tip: set up one and then use Copy!)

In the Survey Flow:

• Set up random assignment, as described above
• Add a Branch Logic element underneath
  • Set the conditional:
    Embedded Data → [name of condition Embedded Data field] → is equal to → [value of condition Embedded Data]
    (e.g., [condition] is equal to [apple])
• Duplicate the Branch Logic, so you have as many as you have conditions
  • Change the condition of the Branch Logic, so that you end up with one Branch Logic per condition
• Place/Move the appropriate Blocks within the matching Branch Logic element

See the screenshots below.

Screenshot: Branch Logic approach — How to set up the Blocks in the Builder

Screenshot: Branch Logic approach — How to set up the Randomizer and Branch Logic in the Survey Flow

Screenshot: Branch Logic approach — Expanded Condition Menu for Branch Logic Element

‹ › ⤢

As you can see, the Survey Flow will accurately represent the design of your experiment.

Note that this will result in a "checkerboard" pattern of "missing" data in your Data File, since each Question you create will have its own column in your Data File, and participants only see one of the Questions. This can be either a pro or a con, depending on the statistical analysis software you use and your personal preferences.

Display Logic

For the Display Logic approach, we will use Display Logic on the Questions themselves (in the Builder).

In the Survey Flow:

• Set up random assignment, as described above

In the Builder:

• Fully configure a Block for one of your conditions
  • Add Display Logic to the Question, using name of the condition
    Display logic → Embedded Data → [field name] → is equal to → [value]
  • Copy the Question and adjust the:
    • Question Name
    • Question Text (and/or Response Option Text)
    • Display Logic condition
  • Repeat the previous step for each condition
  • Leave the Questions so they are within the same Block

Back in the Survey Flow:

• Make sure the Block is below the random assignment Randomizer

See the screenshots below.

Screenshot: Display Logic approach — How to set up the Block in the Builder

Screenshot: Display Logic approach — Expanded Condition Menu for Display Logic

Screenshot: Display Logic approach — How to set up the Randomizer in the Survey Flow, with Questions expanded.

‹ › ⤢

Here, you will be reminded of the flow of your experiment while you are in the Builder — you don't have to go to the Survey Flow to remember. But now you can't really tell what's happening in your Survey Flow. Which you prefer is really up to you.

Note that even though there are multiple questions in the Block, each participant only sees one of them, so the RT data will be unaffected.

Like using Branch Logic, this will result in a "checkerboard" pattern of "missing" data in your Data File, since each Question you create will have its own column in your Data File, and participants only see one of the Questions.

Piped Text

For this approach, we only create one Question and alter the text using Piped Text.

In the Survey Flow:

• Set up random assignment, as described above

In the Builder:

• Fully configure a Block for one of your conditions, except leave out any text that differs between the conditions
• Add Piped Text to the Question, using either of these methods:
  • Click where you want to add text → Click "Piped Text..." → Embedded Data Field → [field name]
  • Type ${e://Field/[field name]} directly in the question where you want the text to appear, replacing [field name] with the name of your condition Embedded Data

Back in the Survey Flow:

• Make sure the Block is below the random assignment Randomizer

See the screenshots below.

Screenshot: Piped Text approach — How to set up the Block in the Builder

Screenshot: Piped Text approach — How to set up the Survey Flow

‹ › ⤢

In the above example, we used the already made condition Embedded Data. If you Question Text is more complicated, you can add fields to the condition Embedded Data. This ensures the text is associated with the right condition. See below.

Screenshot: Piped Text approach — How to set up the Block in the Builder for more complicated differences between conditions.

Screenshot: Piped Text approach — How to set up the Survey Flow for more complicated differences between conditions.

‹ › ⤢

Since you are using the same Question and just altering the text that is displayed, your Data File will only contain one column per Question, no matter how many conditions you have.

Your Questions may be difficult to understand while you are in the Builder, making proofreading and double-checking more difficult.

You also won't be able to see the different wording while you are in the Builder.

Because the Piped Text is created before participants see the Block, previewing your Block will not show the Piped Text. You will have to go through your survey from the beginning.

JavaScript

The JavaScript approach is basically emulating the Piped Text approach yourself, with more flexibility.

In the Survey Flow:

• Set up random assignment like before, but add __js_ to the beginning of the field names

In the Builder:

• Create a question, using the complete text from one of your conditions
• (Optional) Go into HTML View and wrap the text that will differ between conditions in span tags and give the span an ID, like this:
  <span id="fruit-condition">apples</span>
• Add JavaScript to the Question
  Builder Panel → Question behavior → JavaScript

  In the both of the code blocks below, replace condition in getJSEmbeddedData("condition") to be the field name of your condition Embedded Data element

  • If using HTML:
    add this to the addOnload function, but change the fruit-condition to be whatever you named your span in the Question Text
    Copy var conditionSpan = this.getQuestionContainer().querySelector("#fruit-condition"); if (conditionSpan) { conditionSpan.innerText = Qualtrics.SurveyEngine.getJSEmbeddedData("condition"); }
  • If not using HTML:
    add this to the addOnload function, but change the "Do you like " + conditionText + "?" to fit your question.
    Note that this will replace the entire Question text, so you have to reproduce it exactly as you want it.
    Copy var conditionText = Qualtrics.SurveyEngine.getJSEmbeddedData("condition"); if (conditionText) { this.getQuestionTextContainer().innerText = "Do you like " + conditionText + "?"; }

Screenshot: JavaScript approach — How to set up the Block in the Builder

Screenshot: JavaScript approach — How to set up the Question's HTML

Screenshot: JavaScript approach — How to set up the JavaScript

Screenshot: JavaScript approach — How to set up the Survey Flow
Note that the field names all begin with __js_

‹ › ⤢

Unlike the Piped Text approach, here you can set up default (or "fallback") text, so you can see something more sensible in the Builder and when you preview the Block (although like the Piped Text approach, you will have to go through your survey from the beginning for the randomization to occur).

Using JavaScript also allows you to make more complicated changes between conditions, such as changing the colour of the text, along with the text itself or even the URL of an image. Set up the HTML as before, but add more fields to your Embedded Data and add code to the JavaScript for the Question. See below.

Screenshot: JavaScript approach — How to set up the Survey Flow for more complex differences between conditions
Note that the field names all begin with __js_

Screenshot: JavaScript approach — How to set up the JavaScript

Screenshot: JavaScript approach — What the questions will look like

‹ › ⤢

Although it's not strictly necessary to add fields to your Embedded Data with what you want to change, I think it's a good habit to get into. Only being able to see what you changed between conditions when viewing the JavaScript on a specific question can be a little annoying. Having all the information available when you're in Survey Flow can be useful.

This will create extra columns in your Data File (each field of an Embedded Data Element will get its own column) but it's easy enough to remove them.

How to Set Up Order Conditions

This section explains how to set up conditions that are based on order, rather than the content of the items.

Order Conditions

As mentioned previously, if you want to randomly assign participants to see items in a specific order, you would set that up like you would a normal between-subjects condition: creating an Embedded Data element to represent each order and setting up random assignment.

For example, consider if you wanted to randomly assign participants to either see the "forward" order of a scale or the "reverse" order of the scale.

Counterbalancing like this — between subjects — is often done in place of full randomization if there are particular order effects you want to watch our for or are interested in.

Full randomization has its benefits but the downside is that you will lose power the more items you are randomizing, if you are interested in specific order effects. If there are 10 items, you need substantially more participants to analyze a specific item in a specific placement (such as being the first item). So, it depends what you are interested in studying.

For order conditions like this, you would set up the Blocks like specified in the Branch Logic section above (one Block for each Question, since you want participants to see all of the Questions).

But in the Survey Flow, the Branch Logic would contain all of the Blocks, in the matching order.

Tip: Set up one one Branch Logic completely and use Duplicate. Then you just have to adjust the:

• Branch Logic condition
• The order of the Blocks within the Branch Logic

Screenshot: Branch Logic approach — How to set up the Randomizer and Branch Logic in the Survey Flow for Order Conditions

⤢

Counterbalancing

If you are counterbalancing the order of two entire scales — each of which has their own item randomization — you will want to capture which scale a participant saw first. This is where the Groups trick mentioned in the Survey Flow Elements — Groups section comes in.

Note that this "trick" works for any case where you aren't randomizing Blocks or Embedded Data directly, typically when you want to randomize Randomizers. We do this to help make our Data File more readable, since Randomizers and Groups that are being randomized will be referred to by their Flow ID in the Data File.

So, although this is not strictly necessary (unless you want to refer to this order later in your Survey Flow), it's easy to set up and removes the need to fix your Data File later.

However, it gets way more complicated passed two scales, so at that point it might be better to use JavaScript or just fix up your Data File later.

To counterbalance two scales, say CRT and REF, we will utilize the "Groups" element.

1. Create an Embedded Data element — this will store the order
   1. Give it a name, such as crt_ref_order
   2. Set the value to something like none
2. Create a Randomizer directly below
3. Create a Group inside the Randomizer
   1. Give it a name, such as CRT
   2. Create a Branch Logic Element inside the Group
      1. Using the Embedded Data name and value you created in step 1, set the condition to:
         Embedded Data [crt_ref_order] is equal to [none]
      2. Duplicate the Embedded Data Element from step 1
         1. Change the value to represent the order (like crt|ref, in this example)
         2. Place it inside the Branch Logic
4. Duplicate the Group
   1. Change the value of the Embedded Data that's within the Branch Logic to be ref|crt
   2. Change the name of the Group to be REF
5. In each Group, below the Branch Logic, add the Randomizer which is randomizing the elements of the matching scale
6. Ensure that the Randomizer is randomizing 2 elements: the 2 Groups
   1. Make sure the number of elements to present = 2

Screenshot: How to counterbalance the order of items in the Survey Flow, using Branch Logic to capture the order in an Embedded Data element.

⤢

The logic is pretty simple: whichever Group is the first to be shown to participants will update the crt_ref_order Embedded Data. The second Group will find that order is no longer none and will skip the update, preserving the record of which scale came first. We're basically programming using Survey Flow elements, which is pretty neat!

The Embedded Data here is making up for the fact that both Randomizers and Groups that are being randomized will appear in the Data File as their Flow IDs. This is my little trick to deal with that (until Qualtrics fixes this anyway — they should really make the Groups appear as their names, like they do for Blocks).

This also mean you can refer to this later, if needed. For example, if you want to counterbalance a second set of items, based on the order they had for the first set of items.

It might look like a lot, but it really doesn't take that much effort. Particularily because we are utilizing the Duplicate feature. And it's much less annoying than having to deal with data that looks like the column on the right below, which is what you would get if you just randomized Randomizers.

Common Gotchas

Qualtrics is a powerful platform, but it has a few quirks that can catch you off guard — sometimes silently, with no warning. This section collects the most important ones in one place so you know what to watch out for.

• Advanced Randomization: breaks silently when you use the Edit Multiple Menu afterwards
  If you set up Advanced Randomization and then edit any of the option text using the Edit mutilple menu, the randomization will break — and Qualtrics will not warn you. Always set up Advanced Randomization as your very last step, after all option text is finalized. If you need to edit option text afterwards, re-check your Advanced Randomization settings before launching.
• Matrix Tables: Recode Values reset when you use the Edit Multiple Menu
  If you have set up Recode Values for a Matrix Table question and then edit your statements or scale points using the Edit Multiple Menu, Qualtrics will silently revert your Recode Values (the Question Export Tags and/or Variable Naming) back to the default. Always double-check your Recode Values after editing a matrix table.
• Refresh the page after editing Look and Feel
  After making any changes in Look and Feel, refresh the page when you return to the Builder. Skipping this step can cause unexpected behaviour in the Builder.
• Be cautious when multiple people are editing simultaneously
  Qualtrics does not handle simultaneous editing well. If multiple collaborators are editing the survey at the same time, changes may conflict or behave unexpectedly. Try to coordinate with your team so only one person is editing at a time.
• Be cautious when using HTML in question text
  If you edit the question text in the normal (non-HTML) view after inserting your HTML, you may inadvertently delete part of your HTML without realizing it. Switch to HTML view before editing the text of questions with HTML.
• Extra whitespace in question text is often caused by hidden HTML
  If you notice unexpected whitespace at the end of a question, switch to HTML view and check for auto-inserted elements like <p>, <br>, or <div> that Qualtrics may have added. These are easy to miss in the normal view.

Pre-Launch Checklist

Before you distribute your survey, work through the following checklist.

Downloading & Managing Your Data

Once data collection is complete — or even during, if you want to check your results so far — you can export your Data File from Qualtrics.

Exporting Your Data

To export your data, go to Data & Analysis in the top menu → Export & Import → Export Data. This will open the export dialog.

Labels are human-readable but harder to analyze statistically, while Values are easier to work with in R/SPSS but meaningless without the codebook.

Screenshot: Data Download Menu

Screenshot: Data Download Menu with More Options Menu Expanded

‹ › ⤢

Opening Your Data File

To avoid this, consider opening your CSV in Google Sheets instead, which does not auto-convert text in the same way. If you prefer to use Excel, use the Data Import Wizard (File → Import) which allows you to specify the format of each column before opening — though this can be cumbersome for large files with many columns.

Understanding Your Data File

Before you export your data, it helps to know what you're looking at. Qualtrics data files can look intimidating at first — this section walks you through the structure so you're not caught off guard.

File Structure

When you export your data as a CSV file, it will have the following structure:

Important Metadata Columns

Qualtrics automatically includes several metadata columns in every export. The most important ones are:

Cleaning Your Data

Before analysis, remove the following rows from your working file:

Links

Just an organized list of all the links in this guide, for easy reference.

Qualtrics

• Prevent Ballot Box Stuffing
• Saving Your Survey
• Timing Questions
• Custom Validation Messages
• Qualtrics JavaScript Question API Class
• Qualtrics Built-In Embedded Data Fields

Prolific

• URL Parameters
• Setting up Completion Paths

CloudResearch

• URL Parameters
• How to set up an End of Survey message with a Secret Key

MTurk

• Setting up Completion Codes using a Random Number Generator

CSS Code Bank

Append these to the beginning of a CSS ruleset.

• Target a specific Question using its internal Qualtrics ID:
  Copy #question-QID
• Target multiple specific Questions using their internal Qualtrics IDs:
  Copy :is(#question-QID1, #question-QID2, #question-QID3, ..., #question-QIDN)
• Exclude specific Question(s) using their internal Qualtrics ID(s):
  Copy :not(#question-QID1, #question-QID2, #question-QID3, ..., #question-QIDN)

General Rulesets

• Change the font colour of the question text and progress bar % (if you have one):
  Copy .question .question-display, .question-content, #progress-bar-percent { color: #232323; }
• Change the border colour of the radio buttons:
  Copy .question .radio-button.radio { border-color: #5c5c5c; }
• Add space below the progress bar (if you have one):
  Copy #progress-bar-fill { margin-bottom: 12px !important; }

Rulesets for Inserting Text Around Textboxes

• These are user-defined classes for the span elements that will be injected around the textboxes with JavaScript.
  Copy span.spanBefore, span.spanAfter { white-space: pre; vertical-align: middle; } span.spanAfter { margin-left: 6px; } span.spanBefore { margin-right: 6px; }

Question Type Rulesets

Optional CSS rulesets that apply to all Questions of a specific Question Type.

Multiple Choice Questions

• Add an outline to the choices:
  Copy .question.mc .choice-label { outline: 2px solid #eaeaea; }
• Increase the minimum width of horizontally aligned choices:
  Copy .question.mc.horizontal .choice-label { min-width: 5.75rem !important; }
• Increase the minimum width of choices on Mobile:
  Copy @media (max-width: 540px) { .question.mc:is(.horizontal, .vertical) .choice-label { min-width: 7rem !important; } }

Matrix Tables

• Add padding to the element above Matrix Tables:
  Copy .question.matrix.table .question-display-wrapper { padding-bottom: 20px; }
• Give Matrix Tables a rounded outline and adjust their position:
  Copy .question.matrix.table .matrix-content { outline: 0.2rem solid #eaeaea; border-radius: 0.5rem; margin-left: 24px; }
• Preserve the whitespace of the scale point labels:
  Copy .question.matrix.table .matrix-scale-points-item { white-space: pre !important; }
• Shade every other row of Matrix Tables:
  Copy .question.matrix.table .matrix-content > :where(.matrix-row:nth-child(odd)) :is(.matrix-statement-header, .statement-wrapper) { background: color-mix(in srgb, #eaeaea 50%, transparent); }
  Warning: You will want to apply this only to Matrix Tables with 3+ statements. Use the :is() or :not() selectors with the internal Qualtrics ID of the Questions you want to include/exclude from this ruleset.
• Shade every other row of Matrix Tables using :has():
  Copy .question.matrix.table .matrix-content:has( > :nth-child(5)) > :where(.matrix-row:nth-child(odd)) :is(.matrix-statement-header, .statement-wrapper) { background: color-mix(in srgb, #eaeaea 50%, transparent); }
  Warning: This is an alternative to the above, using the amazing :has() selector to only apply to Matrix Tables with 3+ statements (so you don't have to include/exclude any yourself). However, this is very new and will not work for people using outdated browsers. Use it with caution.

Matrix Tables on Mobile

• Add some padding to the question text, to improve the alignment:
  Copy .question.matrix.table .matrix-accordion .foldout-header { padding-left: 1rem; }
• Add some space between the questions, to improve readability:
  Copy .question.matrix.table .matrix-accordion .matrix-statement.without-summary:not(:last-child) { margin-bottom: 2rem; }
• Give Matrix Tables a rounded outline:
  Copy .question.matrix.table .matrix-statement.without-summary { outline: 0.2rem solid #eaeaea; border-radius: 0.5rem; }
• Give the choices an outline:
  Copy .question.matrix.table .matrix-accordion .likert-input { outline: 2px solid #eaeaea; }
• Shade every other choice of a Matrix Question:
  
  Copy .question.matrix.table .matrix-accordion .foldout-items .likert-input:nth-child(odd):not(:hover) { background: color-mix(in srgb, #eaeaea 50%, transparent); }
  Warning: You will want to apply this only to Matrix Tables with 3+ scale points. Use the :is() or :not() selectors with the internal Qualtrics ID of the Questions you want to include/exclude from this ruleset.
• Shade every other choice of a Matrix Question using :has():
  Copy .question.matrix.table .matrix-accordion .foldout-items:has( > :nth-child(3)) .likert-input:nth-child(odd):not(:hover) { background: color-mix(in srgb, #eaeaea 50%, transparent); }
  Warning: This is an alternative to the above, using the amazing :has() selector to only apply to Matrix Tables with 3+ scale pointss (so you don't have to include/exclude any yourself). However, this is very new and will not work for people using outdated browsers. Use it with caution.

Text Entry Questions

• Align the caret to the right:
  Copy .question.te .text-input { text-align: right; }

Single Line

• Set the width of the textbox and adjust padding and text alignment:
  Copy .question.te .text-input:not(.multi-line) { width: 55px !important; padding: 5px !important; vertical-align: middle !important; }

Multiple Lines & Essay Text Box

• Set the width of the textbox and adjust padding and text alignment:
  Copy .question.te .text-input.multi-line { max-width: 100% !important; min-width: 150px !important; min-height: 40px !important; padding: 8px 5px 6px 5px !important; }
• Set the width of the textbox to be the width of the screen for Mobile:
  Copy @media (max-width: 420px) { .question.te .text-input.multi-line { min-width: 100% !important; } }

Form Fields

• Add padding to the element above Form Fields:
  Copy .question.form .question-display-wrapper { padding-bottom: 20px; }
• Increase the font size for the labels and add some padding:
  Copy .question.form ul.choices .choice-label { padding-top: 5px; font-size: 1.25rem; }
• Switch the flex-direction, so the label will be in front of the text box:
  Copy .question.form ul.choices .choice-content { flex-direction: row !important; }
• Set the width of the textbox and adjust padding and text alignment
  Copy .question.form .text-input { text-align: right; width: 55px !important; padding: 5px !important; vertical-align: middle !important; }
  Note: This is identical to the styling used for the Single Line Text Entry Questions, so it's best used for numeric only responses.

Individual Question Rulesets

Optional CSS rulesets for use with individual Questions. Note that these will need to be adjusted for your specific question and you will need to use the internal Qualtrics ID.

Matrix Tables

• Fix how much space the statements take up
  • Adjust the width (900px) of the Matrix Table
  • Adjust the max % of the width (55%) the statements take up
  Copy .question.matrix.table .matrix-content { grid-template-columns: minmax(6.25rem, 55%) 1fr !important; max-width: 900px !important; }

Form Fields

• Align the textboxes
  • Adjust the width (120px) of the labels
  Copy .question.form .choice-label label { width: 120px; }

JavaScript Code Bank

All the JavaScript used in this guide.

Add CSS to a Question

The CSS rulesets used here are identical to the ones in the CSS Code Bank. The difference is that you are placing this code in the JavaScript section of individual Questions, rather than adding it to the custom CSS in Look and Feel. For these you don't need to know the internal Qualtrics ID because the code will take care of that for you.

Matrix Table Questions

• Fix how much space the statements take up
  • Adjust the width (900px) of the Matrix Table
  • Adjust the max % of the width (55%) the statements take up
  • Add to addOnload
  Copy var style = document.createElement("STYLE"); style.innerHTML = ` #question-` + this.questionId + `.question.matrix.table .matrix-content { grid-template-columns: minmax(6.25rem, 55%) 1fr !important; max-width: 900px !important; } `; this.getQuestionContainer().insertAdjacentElement("beforebegin", style);

Form Field Questions

• Align the textboxes
  • Adjust the width (120px) of the labels
  • Add to addOnload
  Copy var style = document.createElement("STYLE"); style.innerHTML = ` #question-` + this.questionId + `.question.form .choice-label { width: 120px !important; } `; this.getQuestionContainer().insertAdjacentElement("beforebegin", style);
  Warning: This assumes you have the general Form Field CSS rulesets above, which changes the alignment of the labels and the textboxes.

General Code

• General code to apply a CSS ruleset to a specific question
  • Replace [REST OF SELECTORS] with the selectors from the ruleset
  • Replace property: value; with the declaration block (everything between the {}) of the ruleset
  • Add to addOnload
  Copy var style = document.createElement("STYLE"); style.innerHTML = ` #question-` + this.questionId + ` [REST OF SELECTORS] { property: value; } `; this.getQuestionContainer().insertAdjacentElement("beforebegin", style);
  Warning: Do not change the string concatenation to template literals (${...}). Template literals (${...}) cannot be used because Qualtrics interprets ${...} as its own Piped Text syntax (I believe, anyway).

Other JavaScript

Functional (non-CSS related) JavaScript.

Add Text Before/After Text Boxes

Text Entry Questions

• If you have the .spanBefore and .spanAfter rulesets in the custom CSS in Look and Feel
  • Add to addOnload
  Copy // whitespace will be preserved, so if you need to, // you can add spaces to adjust the space between the label and the textbox // Insert your text between the quotation marks "" in the 2 lines below: var beforeText = ""; var afterText = ""; var input = this.getQuestionContainer().querySelector(".text-input"); if (beforeText) { var beforeSpan = document.createElement("SPAN"); beforeSpan.innerText = beforeText; beforeSpan.classList.add("spanBefore"); input.parentElement.insertBefore(beforeSpan, input); } if (afterText) { var afterSpan = document.createElement("SPAN"); afterSpan.innerText = afterText; afterSpan.classList.add("spanAfter"); input.after(afterSpan); }
• If you don't have the .spanBefore and .spanAfter rulesets in the custom CSS in Look and Feel
  • Add to addOnload
  Copy // whitespace will be preserved, so if you need to, // you can add spaces to adjust the space between the label and the textbox // Insert your text between the quotation marks "" in the 2 lines below: var beforeText = ""; var afterText = ""; var input = this.getQuestionContainer().querySelector(".text-input"); if (beforeText) { var beforeSpan = document.createElement("SPAN"); beforeSpan.innerText = beforeText; beforeSpan.style.whiteSpace = "pre"; beforeSpan.style.verticalAlign = "middle"; beforeSpan.style.marginRight = "6px"; input.parentElement.insertBefore(beforeSpan, input); } if (afterText) { var afterSpan = document.createElement("SPAN"); afterSpan.innerText = afterText; afterSpan.style.whiteSpace = "pre"; afterSpan.style.verticalAlign = "middle"; afterSpan.style.marginLeft = "6px"; input.after(afterSpan); }

Form Field Questions

• If you have the .spanBefore and .spanAfter rulesets in the custom CSS in Look and Feel
  • Add to addOnload
  Copy // whitespace will be preserved, so if you need to, // you can add spaces to adjust the space between the label and the textbox // insert your text between the quotation marks "" in the line below: var afterText = ""; var inputs = this.getQuestionContainer().querySelectorAll(".text-input"); Array.from(inputs).forEach(input => { var afterSpan = document.createElement("SPAN"); afterSpan.innerText = afterText; afterSpan.classList.add("spanAfter"); input.after(afterSpan); });
• If you don't have the .spanBefore and .spanAfter rulesets in the custom CSS in Look and Feel
  • Add to addOnload
  Copy // whitespace will be preserved, so if you need to, // you can add spaces to adjust the space between the label and the textbox // insert your text between the quotation marks "" in the line below: var afterText = ""; var inputs = this.getQuestionContainer().querySelectorAll(".text-input"); Array.from(inputs).forEach(input => { var afterSpan = document.createElement("SPAN"); afterSpan.innerText = afterText; afterSpan.style.whiteSpace = "pre"; afterSpan.style.verticalAlign = "middle"; afterSpan.style.marginRight = "6px"; input.after(afterSpan); });

Auto-Focus the Text Box

• Pages with One Question

  • Add to addOnReady
  Copy setTimeout(() => { const input = this.getQuestionContainer().querySelector('.text-input'); if (input) input.focus(); }, 0);

• Pages with Multiple Questions

  • Add to addOnReady of one Question
  Copy setTimeout(() => { const input = document.querySelector('.question .text-input'); if (input) input.focus(); }, 0);

Auto-Submit

• Pages with One Question

  • Replace everything in the JavaScript Editor with this:
  Copy (function() { // global variables var container; // question container var handleChange; // function to handle change event Qualtrics.SurveyEngine.addOnload(function() { /*Place your JavaScript here to run when the page loads*/ container = this.getQuestionContainer(); var choices = Array.from(container.querySelectorAll(":is(.choices, .matrix-row)")); // listens for radio buttons to be selected / checkboxes to be checked (i.e., "change" event) // immediately submits the page if an option has been selected handleChange = () => { if (choices.every(choice => choice.querySelector("span.selected"))) { const btn = document.querySelector("#next-button"); if (btn) btn.click(); } }; container.addEventListener("change", handleChange); }); Qualtrics.SurveyEngine.addOnReady(function() { /*Place your JavaScript here to run when the page is fully displayed*/ }); Qualtrics.SurveyEngine.addOnUnload(function() { /*Place your JavaScript here to run when the page is unloaded*/ // remove the listener if (container && handleChange) { container.removeEventListener("change", handleChange); } }); })();
  Note: You can add additional code to the addOnload, addOnReady, or addOnUnload functions like you normally would. The key here is that everything is wrapped in an IIFE (function() { })();. So, just move any existing code into the correct function after copying and pasting the above.
  Warning: Although you might be tempted to use the built-in function this.clickNextButton() (and may even think it's odd that I didn't) I urge you not to — extensive testing has shown me that this.clickNextButton() is unreliable. I have had it fail using the same browser on different computers. Simulating a click using .click() is much more reliable.

• Pages with Multiple Questions

  • Questions must have radio buttons or checkboxes.
    Replace everything in the JavaScript Editor of one Question with this:
  Copy (function() { // global variables var container; // question container var handleChange; // function to handle change event Qualtrics.SurveyEngine.addOnload(function() { /*Place your JavaScript here to run when the page loads*/ container = document.querySelector(".questions-container"); // listens for radio buttons to be selected / checkbox to be checked (i.e., "change" event) // immediately submits the page if an option has been selected handleChange = () => { var choices = Array.from(container.querySelectorAll(".question:not(.hidden) :is(.choices, .matrix-row)")); if (choices.every(choice => choice.querySelector("span.selected"))) { const btn = document.querySelector("#next-button"); if (btn) btn.click(); } }; container.addEventListener("change", handleChange); }); Qualtrics.SurveyEngine.addOnReady(function() { /*Place your JavaScript here to run when the page is fully displayed*/ }); Qualtrics.SurveyEngine.addOnUnload(function() { /*Place your JavaScript here to run when the page is unloaded*/ // remove the listener if (container && handleChange) { container.removeEventListener("change", handleChange); } }); })();
  Note: You can add additional code to the addOnload, addOnReady, or addOnUnload functions like you normally would. The key here is that everything is wrapped in an IIFE (function() { })();. So, just move any existing code into the correct function after copying and pasting the above.
  Warning: Although you might be tempted to use the built-in function this.clickNextButton() (and may even think it's odd that I didn't) I urge you not to — extensive testing has shown me that this.clickNextButton() is unreliable. I have had it fail using the same browser on different computers. Simulating a click using .click() is much more reliable.