JmuTestCase

JMUTestCase class and related code.

The JMUTestCase is a subclass of unittest.TestCase with the following features:

  • Some useful assertions for autograding.

  • Guaranteed test method execution order based on definition order.

  • @required annotation that can be used to make a particular test a requirement for all subsequent tests.

class jmu_gradescope_utils.jmu_test_case.JmuTestCase(methodName='runTest')

Test methods declared within subclasses will be executed in the order they are declared as long as the sortTestMethodUsing attribute of the defaultTestLoader has been set:

unittest.defaultTestLoader.sortTestMethodsUsing = test_compare

They will also respect the @required decorator.

class jmu_gradescope_utils.jmu_test_case.OrderAllTestsMeta(name, bases, local)

Decorate every method so that they will be ordered during testing, and they will respect the ‘required’ decorator.

class jmu_gradescope_utils.jmu_test_case._JmuTestCase(methodName='runTest')

Additional useful assertions for grading.

This is the superclass for JmuTestCase. Users should subclass JmuTestCase in their test code.

assertDocstringsCorrect(filename)

Assert that there are no formatting errors as discovered by flake8.

This will use the config file docstring.cfg included in the autograder folder.

Parameters

filename (str) – The name of the Python file to test

Raises

AssertionError – If flake8 produces any output.

assertInOutput(filename, string_in, expected, variables=None, args='', msg=None, processor=None, from_files=False)

Assert script output contains the indicated string.

See assertOutputEqual() for description of arguments.

assertMatchCount(filename, regex, num_matches, msg=None)

Assert that the regex matches exactly the correct number of times.

Ignores comments and docstrings.

Could be used if the problem instructions say something like: “Your program must use exactly one while loop.”

Parameters

  • filename (str) – The name of the Python file to test

  • regex (str) – A Python regular expression.

  • num_matches (str) – The expected number of matches.

  • msg (str) – Error message that will be printed if the assertion fails.

Raises

AssertionError – If the count doesn’t match.

assertNoConditionals(filename, msg=None)

Assert that the provided script has no conditional statements.

Comments will be ignored. if __name__ == "__main__": will be ignored.

Parameters

  • filename (str) – The name of the Python file to test

  • msg (str) – Error message that will be printed if the assertion fails.

Raises

AssertionError – If the file contains an if.

assertNoForLoops(filename, msg=None)

Assert that the provided script has no for loops.

Comments will be ignored.

Parameters

  • filename (str) – The name of the Python file to test

  • msg (str) – Error message that will be printed if the assertion fails.

Raises

AssertionError – If the file contains a for loop.

assertNoLoops(filename, msg=None)

Assert that the provided script has no for or while loops.

Comments will be ignored.

Parameters

  • filename (str) – The name of the Python file to test

  • msg (str) – Error message that will be printed if the assertion fails.

Raises

AssertionError – If the file contains a loop.

assertNoWhileLoops(filename, msg=None)

Assert that the provided script has no while loops.

Comments will be ignored.

Parameters

  • filename (str) – The name of the Python file to test

  • msg (str) – Error message that will be printed if the assertion fails.

Raises

AssertionError – If the file contains a while loop.

assertNotInOutput(filename, string_in, expected, variables=None, args='', msg=None, processor=None, from_files=False)

Assert script output does not contain the indicated string.

See assertOutputEqual() for description of arguments.

assertOutputCorrect(filename, string_in, expected, variables=None, processor=None)

Wrapper for assertOutputEqual.

I’m not sure why this exists. -NRS

assertOutputEqual(filename, string_in, expected, variables=None, args='', msg=None, processor=None, from_files=False)

Assert correct output for the provided Python script.

Parameters

  • filename (str) – The name of the Python file to test

  • string_in (str) – A string that will be fed to stdin for the script

  • expected (str) – Expected stdout

  • variables (dict) – A dictionary mapping from variable names to values. The script will be edited with these substitutions before it is executed.

  • args (str) – Command line arguments that will be passed to the script.

  • msg (str) – Error message that will be printed if the assertion fails.

  • processor (func) – A function mapping from string to string that will process the script output before it is compared to the expected output.

  • from_files (bool) – Interpret string_in and expected as a file names rather than strings. The files should be stored in the scaffolding folder.

Raises

AssertionError – If the expected output doesn’t match the actual output.

assertOutputNotEqual(filename, string_in, expected, variables=None, args='', msg=None, processor=None, from_files=False)

Assert script output is NOT equal to the indicated string.

See assertOutputEqual() for description of arguments.

assertPassesPep8(filename)

Assert that there are no formatting errors as discovered by flake8.

This will use the config file flake8.cfg included in the autograder folder.

Parameters

filename (str) – The name of the Python file to test

Raises

AssertionError – If flake8 produces any output.

assertRequiredFilesPresent(required_files)

Assert that all files in the provided list were submitted.

Note that this assertion won’t get a chance to run if the test file attempts to import a missing file. One workaround is to do the imports inside the test methods.

Parameters

required_files (list) – A list of Python file names.

Raises

AssertionError – If any of the indicated files are missing.

assertScriptOutputEqual(filename, string_in, expected, variables=None, args='', msg=None, processor=None)

Deprecated wrapper for assertOutputEqual().

getScriptOutput(filename, string_in, variables=None, args='', msg=None, processor=None, only_output=False, from_file=False)

Get output for the provided Python script.

Parameters

  • filename (str) – The name of the Python file to test

  • string_in (str) – A string that will be fed to stdin for the script

  • variables (dict) – A dictionary mapping from variable names to values. The script will be edited with these substitutions before it is executed.

  • args (str) – Command line arguments that will be passed to the script.

  • msg (str) – Error message that will be printed if the assertion fails.

  • processor (func) – A function mapping from string to string that will process the script output before it is returned.

  • only_output (bool) – Return only the stdout (rather than also the stderr).

  • from_file (bool) – Interpret string_in as a file name rather than a string. The file should be stored in the scaffolding folder.

Returns

keys include ‘stdout’, and ‘msg’ as well as ‘stderr’ if there there was any output on stderr. ‘stdout’ and ‘stderr’ are the script output and ‘msg’ is a formatted string describing any execution errors as well as the inputs to the script.

Return type

dict

module_count = 0
run_with_substitution(filename, variables, func)

substitute variable values, then load a module and execute the given function func

jmu_gradescope_utils.jmu_test_case._check_required(func)
jmu_gradescope_utils.jmu_test_case._order(f)
jmu_gradescope_utils.jmu_test_case.required()

Used to decorate required test method. If a required method is failed then all of the following methods will fail as well.

jmu_gradescope_utils.jmu_test_case.test_compare(a, b)