Experiments: From Idea to Launch
A typical algorithmic change begins as an idea from one of our engineers about how to improve search. We take a data-driven approach and all proposed algorithm changes undergo extensive quality evaluation before release.
Engineers typically start by running a series of experiments, tweaking small variables and getting feedback from colleagues until they are satisfied and ready to release the experiment to a larger audience.
Search Quality Rating Guidelines
This document is a version of our Search Quality Rater Guidelines, which gives evaluators examples and guidelines for appropriate ratings. The document focuses on a type of rating task called “URL rating.” In this kind of task, the evaluator looks at a search query and a result that could be returned. They rate the relevance of the result for that query on a scale described within the document. Sounds simple, right? As you can see, there are many tricky cases to think through.
Download Now (English only)
Call 954-361-1227 or email support@carcasm.com if you'd like additional information!
