“Gold Standards” are pre-labeled data points that can be interspersed throughout your data. Think of them as the kind of the Waldo in red and white stockings of data labeling: they hide in your data, pretending to be unlabeled data points.
We've made Gold Standards customizable, so you can use them in several different ways:
In this post, we’ll walk through creating a Gold Standard for our toy "Movie Review" dataset that we created in this post.
After you've uploaded your CSV, create a gold standard by navigating to your project page and selecting "Create Gold Standards.”
In the Gold Standard dashboard, you have the option to create a Synthetic Gold Standard or turn an existing data point into a Gold Standard.
Synthetic Gold Standards are data points created and labeled in the dashboard. You might use a Synthetic Gold standard if you don't want to scroll through your data and you already have a task in mind that you want to create as a Gold Standard. If we wanted to create a Synthetic Gold Standard for our movie review, we'd need to come up with a movie, write a review, and provide a screenshot.
To create a non-synthetic Gold Standard, scroll down to a data point and click the blue plus icon.
And select "Create Gold Standards".
You will be taken to the task you are creating as a Gold Standard.
If you have more than one question in a task, you have the option to set each question as a Gold Standard. You can also provide an explanation of the gold standard; workers will see this explanation if they answer incorrectly.
