Creating Challenges

Challenges are the individual tests that make up a larger assessment. A challenge can be very simple, or very complex. Qualified offers both Code and Quiz challenges.

A Code challenge is a programming test where the developer is given starting code and instructions, and expected to create a solution to meet the specified requirements. Each code challenge contains a set of tests that will be used to validate and score a developer's solution.

A Quiz challenge is one or more questions for the developer to answer. These questions can be multiple choice or free-text answers. Free-text answers can be long or short, and include markldown or code formatting.

To access the Challenge Library, click on Challenges at the top of the screen.

Qualified Challenge Library

Challenges List

Your subscription provides you with access to some or all of the curated Qualified challenges. These challenges have been hand created and customized to fit the needs of either developers or educators. They can give you a quick start to building your first assessment, as well as providing a template for building your own challenges.

You can filter this list by choosing from the filters on the left, including searching by text, as well as filtering by difficulty, type, custom buckets, and programming languages. (Not all challenges are available in all languages.)

You can also use this sidebar to change the sort order.

Most challenges include a summary, and you can peek at the instructions for the selected challenge by clicking Show Instructions. If a challenge has multiple languages, use the language tabs on the right to see the instructions for that language, which may include different code samples.

Challenge Details

Clicking on the title for a challenge takes you to the challenge details screen. This let's you view all the information about a challenge, including the tests and a working reference solution (subscription required for complete access). You can also other view the statistics about the historical pass rate for a challenge, and more.

You also have several actions available to you, including:

Team Challenge Library

Challenge Details

Team challenges have similar details available to them as Qualified challenges, with some additional options.

Creating & Editing Challenges

You can create a custom challenge either by using the IMPORT option mentioned above, or you can use the CREATE A CHALLENGE menu to start with a blank slate.

General Info & Settings

This page provides all the general information about the challenge, such as title, summary, difficulty, and estimated time.

If you've imported a library challenge, it will be linked to the library version, and other options won't be editable until you unlink it.

Challenge Instructions

This page provides a side-by-side Markdown editor that allows you to provide detailed instructions to the developer.

More information on Editing Challenge Instructions can be found here.

Code Challenges: Languages

Each challenge needs one or more language sections. You can have as many as you like.

If you are starting from a blank challenge, or want to add a new language, click ADD LANGUAGE to choose a language to add.

This will take you to the Code Challenge Editor.

Code Challenges: Language Generation

Qualified offers the ability to generate code challenge languages from a single configuration file. This works particularly well for algorithmic challenges whose concepts transcend the language used.

More information on Language Generation can be found here.

Quiz Challenges: Questions

This page enables you to add the questions for this Quiz challenge.

For more details, please see Quiz Challenge Questions.

Previewing

You can preview a challenge before saving by using the PREVIEW button, or pressing CTRL + P. This will let you see how the challenge works as a candidate, including previewing scoring results. Pressing CTRL + P again will return you to the editor.

Saving & Publishing

Once you are comfortable with the challenge you have entered, you can choose to SAVE & PUBLISH the challenge immediately. If you aren't ready to exit the editor, you can use the Save Changes option under the save button menu.

If you don't want to publish the challenge, you can keep it in the Draft state by using Save Draft under the save button menu.

For Code Challenges, publishing the challenge will also validate the challenge. This means each language will be submitted and run against your reference solution. This helps to ensure the quality of the challenge, but also enables our system to gather important baseline information, such as the total number of tests and other basic information about the challenge.