As we have now said farewell to 2018, Qualified would like to thank you for being a part of our community. As we at Qualified look forward towards what's to come for 2019, we wanted to share our top 10 favorite features and improvements of 2018. We are happy we get to look back and be able to say we've championed quite a few items that makes it easier to hire Qualified candidates.
10. Drag and Drop Markdown Images We made it easier than ever to add example diagrams and other images to your challenge descriptions, notes, reviews, and nearly anywhere else Markdown is used within the Qualified application (this includes candidate notes about their solutions!).
Just simply drag-and-drop images right into the editor to upload them to our site, and they'll be uploaded and inserted automatically. You can also select files from a file selector, or even use copy-and-paste to add images quickly.
9. Notification Schemes Previously notifications would only be sent to team members who were assigned to the candidate. Some teams however, wanted everyone to receive notifications for every candidate, or only team members with certain roles (such as reviewers). Notification schemes to the rescue! We have added various schemes so that you can now configure the notification style that fits your team the best!
These schemes can be configured at the team level, and overridden on a per assessment basis. By default, we still only notify team members who are assigned to the candidate.
8. Assessment Review Ratings Column We believe that augmenting the auto-graded score with manual reviews conducted by your team is the best way to identity the best talent. We aggregate all overall assessment review ratings and show them within the candidates list. This means you can now have not just an overview of how well your candidates are scoring, but an overview of what your team thinks of each candidate.
To dig a little deeper, you can click on each rating group within a candidate row to display which team members rated the candidate.
7. Text Questions: Validation Matchers Preview Text questions support a rich set of matchers for determining how the answer is scored, including the ability to use multiple regular expressions. Now you can preview what those matchers will match through our new preview tool.
6. Preview Edited Challenges We have updated our platform to allow you to preview the quiz you are editing before it's been saved, so you know what it will look like to the candidate every step of the way. Press Preview on the top left side of the dashboard whenever you want to see what you have edited.
Pro Tip: you can press CTRL/CMD+P to quickly preview a challenge while editing, then press it again to close the preview.
5. Zoomable Quiz Images Candidates can zoom in on large quiz images. This allows you to include images in your quiz questions, such as large diagrams, which candidates can zoom in on to fully understand all of the details. UML diagrams, SQL diagrams, etc are now all easily usable at any size!
4. Weighted Multiple Choice Questions Weighted multiple choice questions work exactly as normal multiple choice questions from the candidate's point-of-view, but now you can assign different values to each question. The total score for the questions can then be assigned anything you want, such as the sum of all correct answers, the highest correct answer, or any value in-between.
By allowing choices to be marked as "more correct" or "less wrong" than others, an entire new set of possibilities opens up! This includes use cases such as aptitude testing, personality testing and job fit surveys.
3. Timed Quizzes & Timed Questions Placing a time limit on a quiz can help a candidate better time manage where they place their effort and help the flow of assessment completion move faster. It also gives you the ability to create an assessment that is about speed, such as adding more questions to an assessment than a candidate can answer in a certain amount of time.
Placing a time limit on a question can be the best way to assess a candidates prior knowledge. It restricts the amount of time a candidate can spend on a specific question, creating a time pressure forces them to think about the answer, instead of giving them time to go look the answer up. There are two variations of this feature: Per Question Time Limit and Per Question Time Limit with Bonus. Time bonuses are a great way of allowing you to give candidates a bit more time to at least get some credit, but rewarding those who answer quickly.
2. Compare Code Solutions In addition to being able to compare a candidate's solution to the reference solution for the challenge, you can now also compare their solutions to those of your team members, as well as other candidates who have submitted a solution.
Reviewing the code quality of submitted solutions is an important step for evaluating a candidate, and now that you can see other solutions, it's much easier to get a relative understanding of how one candidate solves problems compared to others.
1. Code Similarity (Premium Feature) Code Similarity allows us to constantly compare every solution in the system to determine the similarity between newly submitted solutions and older solutions. This comparison reports back a similarity score, as well as a complexity score.
While there are many use cases for code similarity, such as simply surfacing similar but not exact solutions when comparing candidate's code to others, the main application of this feature is to detect possible plagiarized solutions. There is no way for Qualified to automatically say for sure that a solution is plagiarized, since the system only determines that code is similar, it doesn't know why the code is similar. However our system will take certain factors into account, such as complexity, to determine a range in which code being similar starts to be considered a risk of being plagiarized. Our system will identity solutions that may be at risk as either "low", "medium" or "high" risk. From there your team can decide if you believe the solutions to in fact be plagiarized.
That's our top 10 list for this year. Of course we have added many more features and improvements. Head over to our Product Updates page to get a full list.