Distribution Settings

Distribution settings are the last step before a task can be reviewed and published, and it's important to note that they can't be changed after a task is published. Read on to get a sense of how distribution settings should differ with different tasks.

We'll be discussing jobs a lot in this article: a job is the data that is presented to the contributor for response at one time (e.g. one image/one sentence), and more than one contributor can complete the same job. Effectively, the number of responses a job receives is equal to the number of jobs under a task multiplied by the number of contributors who complete each job. This concept is irrelevant for data collection tasks with no input data presented to contributors, as essentially one job is presented to all contributors.

See Terminology for more information.

End date

The end date of a task is the day that the task closes to contributors. It will no longer be available for contributors to complete, regardless of how many responses have been collected.

It does not affect responses that are already collected.

Visibility

Default: Public.

Feature Public Unlisted Private
Listed in "Recommended" only if email address matches
Included in searches if keyword matches task ID only if email address matches
Accessible with shared links only if email address matches
Work on the task eligible contributors only eligible contributors only eligible contributors
with a matching email address only

Requirements set in the Contributors page are always enforced under any task visibility options.

Public

Everyone can see the task, and eligible contributors can work on it.

Unlisted

The task will only be visible to contributors who have the link or task ID. You will have share a direct access link of the task to your contributors, after publishing the task.

Private

Only specific contributors can see and work on this task. You will have provide a list of emails of people who are allowed to view and work on the task. You may share a direct access link of the task to your contributors but it is not required. To them, the task will be visible and searchable just like public tasks.

Allow skipping

Default: Yes.

This option allows contributors to skip a job when they are uncertain of how to respond and still be able to complete other jobs under the task. Contributors won't receive a reward for skipping a job.

It is recommended to allow contributors to skip jobs so that they aren't pushed to make guesses or give random answers.

Response acceptance

Default: Manual.

After a contributor completes a jobs, their response must be accepted before it can be included in your final data set and they can receive a reward for it, meaning you can only export accepted responses. Automatic response acceptance means that every single response will be automatically rewarded and included in your final data set.

We recommend choosing manual response acceptance so that you can have the opportunity to review each response and either accept or reject it. Rejected responses will not be rewarded or included in your final data set.

Job distribution order

Default: Start jobs sequentially, and jobs with more responses will complete first.

The order in which jobs will appear to contributors when they choose to do a task.

There are two points to consider when choosing among the options:

1. Should the jobs start sequentially or randomly?

The difference is not important if you plan to end the task only after all the jobs are completed. If not, you may decide whether you want jobs to be started sequentially (according to the row order of the CSV file, and names of the media resource files), or randomly.

2. Should the jobs be finished one after another, or should all jobs be kept at a similar progress?

By allowing a job to be finished first, it will be more likely to reach the targeted consensus size, so that you can review the responses within the same Review Period.

You may refer to the animations below for the options:

Animation Option
Start jobs randomly, then jobs with more responses finish first
Start jobs randomly, and maintain a similar progress among all jobs
Start jobs sequentially, then jobs with more responses finish first
Start jobs sequentially, and maintain a similar progress among all jobs

Time limit

Default: A duration suggested according to the task template chosen.

The amount of time each contributor has to complete a job.

Once they have had a job open for that amount of time, the job will disappear. This is to ensure that tasks are not being held for indefinite periods of time by one contributor, as this prevents any other contributors from working on them. If the contributor wishes to work on the task again, they will be distributed a new job.

This time limit should generally be short (e.g. 5-10 minutes), unless contributors need time to make video or audio recordings.

Consensus size

Default: A number suggested according to the task template chosen.

The number of responses to the same job.

Once the consensus size is reached, that job will no longer be distributed to contributors.

For tasks without input data (e.g. collecting images of the same subject), the concept of jobs isn't so relevant, as there is essentially only one job that is presented to all contributors. For those tasks, simply enter the number of data responses you'd like to collect. For example, if you're collecting 500 images of something, enter 500 as the consensus size.

Limit per contributor

Default: A number suggested according to the task template chosen.

The maximum number of submissions from a single contributor.

A higher limit means the task will reach consensus size more quickly, but there will be less variety in contributors.

"0" indicates no limit (recommended).

Reward

Rewards are given to contributors, per response, once it is accepted, or when the review period of that response has passed.

The amount entered does not including our 20% publishing fee.

The total cost to you for the entire task will be calculated and shown at the below of the page. After you make this payment, it will be considered credit under your account, so if you close a task before all the rewards you paid for are given out, they will still be available as rewards for other tasks published under your account.