Project Code Challenges

Project code challenges allow you to deeply test a candidate's skill using complete, realistic code bases. Unlike our classic code challenges, these can only be created against a single language, but the advantage is your candidate is able to come up with more advanced solutions that show off their experience.

Project Code Challenges are a new beta feature of Qualified, and may not be available on your account. Please reach out to your sales representative if you'd like to join the beta.

Project-Code-Challenge-Specific Details

Project code challenges have many files per challenge, which you can view inside the challenge preview.

Solving Project Code Challenges

The workflow for solving a project code challenge is:

  1. The candidate reviews the instructions, and provided setup code.
  2. They edit their solution using the online web IDE, modifying editable files, or adding new ones as necessary.
  3. For front-end web challenges, candidates can use the WEB PREVIEW to see real-time rendering for their solution while editing.
  4. At any time, they can use the RUN TESTS button to validate their solution. They can edit the provided test cases if they want to perform additional testing.
  5. Using the SUBMIT SOLUTION button, their code is submitted against the submission test cases. Unlike with classic code challenges, this result is never shown to the candidate.
    • Even if they don't click SUBMIT SOLUTION, Qualified will still detect when they are done with the challenge and process their solution.
  6. While they can't review the submission results, they are allowed to go back and edit their solution and re-submit it as long as the assessment itself has not been submitted. This could be if, for example, they had gotten stuck but thought of a solution and want to complete the challenge.

Project Code Challenge Preview UI

You can open the solver for project challenges either by previewing an entire assessment, or using the PREVIEW button on a challenge.

This solver gives you the ability to test writing solution code, add or modify test cases for validating the solution, and preview the output. As it's a preview, there is also a button to preview the submission test cases, but these will never be shown to the candidate.

Within the preview window, nothing is saved, this is purely a way to quickly view how a challenge works.

Creating Project Code Challenges

You can create a challenge by using the CREATE A CHALLENGE menu in the Challenges sidebar, or you can use the EDIT or DUPLICATE buttons to modify an existing one.

Editor UI

The project editor is more advanced than the classic code editor, and works more like a traditional IDE. You can open and close various pages and files, and re-arrange the UI by dragging tabs around.

Previewing

You can preview a challenge before saving by using the PREVIEW button, or pressing CTRL + P. This will let you see how the challenge works as a candidate, including previewing scoring results. Pressing CTRL + P again will return you to the editor.

Saving & Publishing

Once you are comfortable with the challenge you have entered, you can choose to SAVE & PUBLISH the challenge immediately. If you aren't ready to exit the editor, you can use the Save Changes option under the save button menu.

If you don't want to publish the challenge, you can keep it in the Draft state by using Save Draft under the save button menu.

Publishing the challenge will also validate the challenge. This means each language will be submitted and run against your reference solution. This helps to ensure the quality of the challenge, but also enables our system to gather important baseline information, such as the total number of tests and other basic information about the challenge.

Running Code

The editor can run your code for previewing output against either the candidate tests (those shown to the candidate) or the submission tests (those used for scoring). Use the RUN CANDIDATE TESTS and RUN SUBMISSION TESTS buttons respectively.

Shortcut Key Commands:

Run Fileset

Normally, you'll want to run the tests against the Reference fileset. This will allow you to test that your tests are all working as expected. Occasionally, you'll want to ensure the setup files don't pass the tests. To do this, switch the Run Fileset mode to Project (Setup), which will not include any reference files in the solution.

Challenge Config

Description & Settings

This page provides all the general information about the challenge, such as title, summary, difficulty, and estimated time.

If you've imported a library challenge, it will be linked to the library version, and other options won't be editable until you unlink it.

Instructions

This page provides a side-by-side Markdown editor that allows you to provide detailed instructions to the developer.

Run Configuration

Allows you to view the environment and adjust some properties of the runtime configuration for this challenge, such as enabled services or paths that are ignored during submission.

Project Files

These are all the base files that make up a challenge. These are what are provided to the candidate so they can solve the challenge.

Reference Solution Files

Reference solution files allow the code runner to validate your tests, as well as learn about the expected output from the tests. Any files provided in here are treated as files provided by a candidate, overriding (when allowed) the project files above.

If you want to preview which files are sent to the runner at any time, open the Files Sent to Runner tab.

Challenge Editor Tools

Run Output

Open to see the output from running your code. It will automatically be opened if you use the RUN buttons.

Web Preview

For challenges that target front-end web development, we provide a real-time, in-IDE preview panel. You and the candidate can use this to see how a solution renders.

Please note that Web Previews require a somewhat specific project layout, so you may be restricted in how your challenge is designed. Using one of the Qualified templates is the best way to get started with your front-end challenge.

Files Sent to Runner

This tab shows a listing of which files that will be sent to the runner based on which mode in use (Run Fileset and Candidate vs Submission tests).

IDE Settings

Configure the IDE to your liking here.

IDE Help

Inline help for the IDE.