Challenges are the individual tests that make up a larger assessment. A challenge can be very simple, or very complex. Qualified offers both Code and Quiz challenges.
A Code challenge is a programming test where the developer is given starting code and instructions, and expected to create a solution to meet the specified requirements. Each code challenge contains a set of tests that will be used to validate and score a developer's solution.
A Quiz challenge is one or more questions for the developer to answer. These questions can be multiple choice or free-text answers. Free-text answers can be long or short, and include markldown or code formatting.
To access the Challenge Library, click on Challenges at the top of the screen.
Qualified Challenge Library
Your subscription provides you with access to some or all of the curated Qualified challenges. These challenges have been hand created and customized to fit the needs of either developers or educators. They can give you a quick start to building your first assessment, as well as providing a template for building your own challenges.
You can filter this list by choosing from the filters on the left, including searching by text, as well as filtering by difficulty, type, custom buckets, and programming languages. (Not all challenges are available in all languages.)
You can also use this sidebar to change the sort order.
Most challenges include a summary, and you can peek at the instructions for the selected challenge by clicking Show Instructions. If a challenge has multiple languages, use the language tabs on the right to see the instructions for that language, which may include different code samples.
Clicking on the title for a challenge takes you to the challenge details screen. This let's you view all the information about a challenge, including the tests and a working reference solution (subscription required for complete access). You can also other view the statistics about the historical pass rate for a challenge, and more.
You also have several actions available to you, including:
- IMPORT: Import the challenge into your library, which allows you to get a quick start on building your own custom challenge, or make changes to an existing challenge that might not be quite right for you.
- PREVIEW: Quickly Preview the challenge as a candidate would see it.
- ADD TO ASSESSMENT: Add a challenge directly to an assessment.
Team Challenge Library
Team challenges have similar details available to them as Qualified challenges, with some additional options.
- EDIT: Let's you edit the challenge
- DELETE: Let's you delete unused challenges
- PUBLISHED/DRAFT/ARCHIVED: You can change the state of the challenge
- DUPLICATE Clone the challenge into a new one
- FOLLOW UP CHALLENGE: Create a new challenge based off the solution to this one.
Creating & Editing Challenges
You can create a custom challenge either by using the IMPORT option mentioned above, or you can use the CREATE A CHALLENGE menu to start with a blank slate.
General Info & Settings
This page provides all the general information about the challenge, such as title, summary, difficulty, and estimated time.
This page provides a side-by-side Markdown editor that allows you to provide detailed instructions to the developer.
More information on Editing Challenge Instructions can be found here.
Code Challenges: Languages
Each challenge needs one or more language sections. You can have as many as you like.
If you are starting from a blank challenge, or want to add a new language, click ADD LANGUAGE to choose a language to add.
This will take you to the Code Challenge Editor.
Code Challenges: Language Generation
Qualified offers the ability to generate code challenge languages from a single configuration file. This works particularly well for algorithmic challenges whose concepts transcend the language used.
More information on Language Generation can be found here.
Quiz Challenges: Questions
This page enables you to add the questions for this Quiz challenge.
For more details, please see Quiz Challenge Questions.
You can preview a challenge before saving by using the PREVIEW button, or pressing ⌘CTRL + P. This will let you see how the challenge works as a candidate, including previewing scoring results. Pressing ⌘CTRL + P again will return you to the editor.
Saving & Publishing
Once you are comfortable with the challenge you have entered, you can choose to SAVE & PUBLISH the challenge immediately. If you aren't ready to exit the editor, you can use the Save Changes option under the save button menu.
If you don't want to publish the challenge, you can keep it in the Draft state by using Save Draft under the save button menu.
For Code Challenges, publishing the challenge will also validate the challenge. This means each language will be submitted and run against your reference solution. This helps to ensure the quality of the challenge, but also enables our system to gather important baseline information, such as the total number of tests and other basic information about the challenge.
Challenge VersionsSemantic Versioning system for Qualified Challenges
Challenge: Advanced Test CasesDetails about advanced test reporting features such as hidden, weighted and tagged test cases
Challenge: Carry Forward SolutionsDetails about how to write a challenge that will candidates to pull and modify a previous solution to another challenge.
Challenge: InstructionsHelp for writing Challenge Instructions
Code Challenge Editor IDEReference for editing code challenges in the Qualified online IDE
Code Challenge: Language GeneratorHow to use the Language Generator
Quiz Challenge QuestionsReference for editing the questions on quiz challenges