autograder
The autograder.cli package contains tools for interacting with the autograder.
Each package can be invoked to list the tools (or subpackages) it contains.
Each tool includes a help prompt that accessed with the -h/--help flag.
This package contains the following CLI tools:
This package provides tools for working with project configuration options.
List the current configuration options.
usage: python3 -m autograder.cli.config.list [-h] [--version] [--testing-mode]
[--include-cli] [--show-origin]
[--skip-header]
Update configuration options. The file at the specified config location will be created if it doesn't exist.
usage: python3 -m autograder.cli.config.set [-h] [--version] [--testing-mode]
[--local | --project | --global | --file ]
= [= ...]
Unset the first matching instance of a configuration option. Does nothing if there is no matching config.
usage: python3 -m autograder.cli.config.unset [-h] [--version]
[--testing-mode]
[--local | --project | --global | --file ]
KEY [KEY ...]
The `autograder.cli.courses` package contains tools to access and manage course information.
The `autograder.cli.courses.admin` package contains tools for administering an autograder course.
Send an email to course users.
usage: python3 -m autograder.cli.courses.admin.email [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--course COURSE]
[--dry-run] [--to TO]
[--cc CC] [--bcc BCC]
--subject SUBJECT
[--body BODY] [--html]
[--testing-mode]
Update an existing course using its own source.
usage: python3 -m autograder.cli.courses.admin.update [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--dry-run]
[--skip-emails]
[--skip-source-sync]
[--skip-lms-sync]
[--skip-build-images]
[--skip-template-files]
[--course COURSE]
[--testing-mode]
The `autograder.cli.courses.assignments` package contains tools to work with course assignments.
The `autograder.cli.courses.assignment.images` package contains tools to work with assignment Docker images.
Fetch an assignment's Docker image.
usage: python3 -m autograder.cli.courses.assignments.images.fetch
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT] [--out-dir OUT_DIR]
[--testing-mode]
Get information about an assignment's Docker image (does not include the actual image).
usage: python3 -m autograder.cli.courses.assignments.images.info
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT] [--testing-mode]
Fetch an assignment grading report.
usage: python3 -m autograder.cli.courses.assignments.report
[-h] [--version] [--server SERVER] [--course COURSE] [--user USER]
[--pass PASS] [--testing-mode] [--format {json,table,text}]
[--include-extra-fields] [--pretty-headers] [--skip-headers]
The `autograder.cli.courses.assignments.submissions` package contains tools to work with assignment submissions.
The `autograder.cli.courses.assignments.submissions.analysis` package contains tools to work with assignment analyses.
Get the result of a individual analysis for the specified submissions.
usage: python3 -m autograder.cli.courses.assignments.submissions.analysis.individual
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--dry-run] [--overwrite-records] [--wait-for-completion]
[--testing-mode]
SUBMISSION [SUBMISSION ...]
Get the result of a pairwise analysis for the specified submissions.
usage: python3 -m autograder.cli.courses.assignments.submissions.analysis.pairwise
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--dry-run] [--overwrite-records] [--wait-for-completion]
[--testing-mode]
SUBMISSION [SUBMISSION ...]
The `autograder.cli.courses.assignment.submissions.fetch` package contains tools to query submissions to the autograder.
The `autograder.cli.courses.assignment.submissions.fetch.course` package contains tools to query submissions to the autograder at a course level.
Get all recent submissions and grading information for this assignment.
usage: python3 -m autograder.cli.courses.assignments.submissions.fetch.course.attempts
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-users TARGET_USERS] [--out-dir OUT_DIR] [--testing-mode]
Get the most recent scores for this assignment.
usage: python3 -m autograder.cli.courses.assignments.submissions.fetch.course.scores
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-users TARGET_USERS] [--testing-mode]
[--format {json,table,text}] [--include-extra-fields]
[--pretty-headers] [--skip-headers]
The `autograder.cli.courses.assignment.submissions.fetch.user` package contains tools to query submissions to the autograder at a user level.
Get a submission along with all grading information.
usage: python3 -m autograder.cli.courses.assignments.submissions.fetch.user.attempt
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-email TARGET_EMAIL] [--target-submission TARGET_SUBMISSION]
[--out-dir OUT_DIR] [--testing-mode]
Get all submission attempts made by a user along with all grading information.
usage: python3 -m autograder.cli.courses.assignments.submissions.fetch.user.attempts
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-email TARGET_EMAIL] [--out-dir OUT_DIR] [--testing-mode]
Get the most recent scores for this user and assignment.
usage: python3 -m autograder.cli.courses.assignments.submissions.fetch.user.history
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-email TARGET_EMAIL] [--testing-mode]
[--format {json,table,text}] [--include-extra-fields]
[--pretty-headers] [--skip-headers]
Get a copy of the grading report for the specified submission. Does not submit a new submission.
usage: python3 -m autograder.cli.courses.assignments.submissions.fetch.user.peek
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-email TARGET_EMAIL] [--target-submission TARGET_SUBMISSION]
[--testing-mode]
The `autograder.cli.courses.assignment.submissions.proxy` package contains tools to make proxy submissions to the autograder.
Proxy regrade an assignment for all target users using their most recent submission.
usage: python3 -m autograder.cli.courses.assignments.submissions.proxy.regrade
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-users TARGET_USERS] [--regrade-cutoff REGRADE_CUTOFF]
[--dry-run] [--overwrite-records] [--wait-for-completion]
[--testing-mode]
Proxy resubmit an assignment submission to the autograder.
usage: python3 -m autograder.cli.courses.assignments.submissions.proxy.resubmit
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-submission TARGET_SUBMISSION] --proxy-email PROXY_EMAIL
[--proxy-time PROXY_TIME] [--testing-mode]
Proxy submit an assignment submission to the autograder.
usage: python3 -m autograder.cli.courses.assignments.submissions.proxy.submit
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT] [--message MESSAGE]
--proxy-email PROXY_EMAIL [--proxy-time PROXY_TIME] [--testing-mode]
FILE [FILE ...]
Remove a specified submission. Defaults to the most recent submission.
usage: python3 -m autograder.cli.courses.assignments.submissions.remove
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT]
[--target-email TARGET_EMAIL] [--target-submission TARGET_SUBMISSION]
[--testing-mode]
Submit an assignment submission to the autograder.
usage: python3 -m autograder.cli.courses.assignments.submissions.submit
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT] [--message MESSAGE]
[--allow-late] [--testing-mode]
FILE [FILE ...]
Get a course from a server.
usage: python3 -m autograder.cli.courses.get [-h] [--version]
[--server SERVER]
[--course COURSE] [--user USER]
[--pass PASS] [--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
List the courses on a server.
usage: python3 -m autograder.cli.courses.list [-h] [--version]
[--server SERVER] [--user USER]
[--pass PASS] [--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
The `autograder.cli.courses.lms` package contains tools for interacting with the given course's LMS. All LMS operations are done within the context of the autograder. For more general LMS functionality, see the [LMS Toolkit](https://github.com/edulinq/lms-toolkit).
The `autograder.cli.courses.lms.scores` package contains tools for working with assignment scores for a course's LMS.
Perform a full scoring and upload scores to the course's LMS.
usage: python3 -m autograder.cli.courses.lms.scores.upload [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--course COURSE]
[--dry-run]
[--testing-mode]
The `autograder.cli.courses.stats` package contains tools for accessing course-level statistics.
Query stats for this course.
usage: python3 -m autograder.cli.courses.stats.query [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--course COURSE]
--metric-type
{api-request,code-analysis-time,grading-time,cpu-usage,mem-usage,net-in,net-out,task-time}
[--limit LIMIT]
[--after QUERY_AFTER]
[--before QUERY_BEFORE]
[--sort QUERY_SORT]
[--where KEY=VALUE [KEY=VALUE ...]]
[--testing-mode]
The `autograder.cli.courses.upsert` package contains tools for upserting an autograder course.
Upsert a course using a file specification (FileSpec).
usage: python3 -m autograder.cli.courses.upsert.filespec [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--dry-run]
[--skip-emails]
[--skip-source-sync]
[--skip-lms-sync]
[--skip-build-images]
[--skip-template-files]
--type FILESPEC_TYPE
--path FILESPEC_PATH
[--reference FILESPEC_REFERENCE]
[--username FILESPEC_USERNAME]
[--token FILESPEC_TOKEN]
[--testing-mode]
Upsert a course using a zip file or directory.
usage: python3 -m autograder.cli.courses.upsert.zip [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS] [--dry-run]
[--skip-emails]
[--skip-source-sync]
[--skip-lms-sync]
[--skip-build-images]
[--skip-template-files]
[--testing-mode]
PATH
The `autograder.cli.courses.users` package contains tools to work with course users.
Drop a user from a course.
usage: python3 -m autograder.cli.courses.users.drop [-h] [--version]
[--server SERVER]
[--course COURSE]
[--user USER]
[--pass PASS]
[--target-email TARGET_EMAIL]
[--testing-mode]
Enroll users from a TSV file into a course.
usage: python3 -m autograder.cli.courses.users.enroll-file [-h] [--version]
[--server SERVER]
[--course COURSE]
[--user USER]
[--pass PASS]
[--dry-run]
[--skip-inserts]
[--skip-updates]
[--send-emails]
[--testing-mode]
PATH
Enroll a user in a course.
usage: python3 -m autograder.cli.courses.users.enroll [-h] [--version]
[--server SERVER]
[--course COURSE]
[--user USER]
[--pass PASS]
[--dry-run]
[--skip-inserts]
[--skip-updates]
[--send-emails]
--new-email NEW_EMAIL
[--new-name NEW_NAME]
[--new-course-role {unknown,other,student,grader,admin,owner}]
[--new-lms-id NEW_LMS_ID]
[--testing-mode]
Get information about a course user.
usage: python3 -m autograder.cli.courses.users.get [-h] [--version]
[--server SERVER]
[--course COURSE]
[--user USER] [--pass PASS]
[--target-email TARGET_EMAIL]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
List the users in a course.
usage: python3 -m autograder.cli.courses.users.list [-h] [--version]
[--server SERVER]
[--course COURSE]
[--user USER]
[--pass PASS]
[--target-users TARGET_USERS]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
The `autograder.cli.grading` package contains tools to grade submissions or help in the grading process.
Grade a submission given an already prepared grading directory (see autograder.cli.testing.setup-grading-dir) and a grader file. Use autograder.cli.grading.grade if you have not already prepared your grading directory.
usage: python3 -m autograder.cli.grading.grade-dir [-h] -g GRADER -d DIR
[-o OUTPATH]
Grade an assignment (specified by an assignment JSON file) with the given submission. Non-Python assignments can be graded, but they require an "invocation" field' in the assignment config, and the running machine must be configured to run them' (e.g. have all the required software installed).
usage: python3 -m autograder.cli.grading.grade [-h] [--version]
[--testing-mode]
[-a ASSIGNMENT] -s SUBMISSION
[-o OUT_PATH]
[-t TEST_SUBMISSION_PATH]
Take any necessary steps before performing a standard docker-based grading. This script is used as part of the standard docker grading procedure for Python.
usage: python3 -m autograder.cli.grading.pre-docker [-h] [-c CONFIG]
[-bd BASEDIR]
[-id INPUTDIR]
[-od OUTPUTDIR]
[-wd WORKDIR]
The `autograder.cli.stats` package contains tools for managing stats from the autograder server.
Query log entries from the autograder server.
usage: python3 -m autograder.cli.logs.query [-h] [--version] [--server SERVER]
[--user USER] [--pass PASS]
[--level {TRACE,DEBUG,INFO,WARN,ERROR,FATAL,OFF}]
[--after QUERY_AFTER]
[--past QUERY_PAST]
[--target-course QUERY_TARGET_COURSE]
[--target-assignment QUERY_TARGET_ASSIGNMENT]
[--target-email QUERY_TARGET_EMAIL]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
The `autograder.cli.metadata` package contains tools to access autograder metadata.
Describe all endpoints on the server.
usage: python3 -m autograder.cli.metadata.describe [-h] [--version]
[--server SERVER]
[--force-compute]
[--testing-mode]
Get a heartbeat from the server.
usage: python3 -m autograder.cli.metadata.heartbeat [-h] [--version]
[--server SERVER]
[--testing-mode]
The `autograder.cli.stats` package contains tools for managing stats from the autograder server.
Query stats for the server.
usage: python3 -m autograder.cli.stats.query [-h] [--version]
[--server SERVER] [--user USER]
[--pass PASS] --metric-type
{api-request,code-analysis-time,grading-time,cpu-usage,mem-usage,net-in,net-out,task-time}
[--limit LIMIT]
[--after QUERY_AFTER]
[--before QUERY_BEFORE]
[--sort QUERY_SORT]
[--where KEY=VALUE [KEY=VALUE ...]]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
The `autograder.cli.system` package contains tools for interacting with the system/server running the autograder.
Get stack traces for all the currently running routines (threads) on the server.
usage: python3 -m autograder.cli.system.stacks [-h] [--version]
[--server SERVER] [--user USER]
[--pass PASS] [--testing-mode]
The `autograder.cli.testing` package contains tools to test, debug, and develop courses and assignments.
Generate test data by starting the specified server and running all tests in this project.
usage: python3 -m autograder.cli.testing.generate-test-data
[-h] [--version] [--testing-mode]
[--server-output-file SERVER_OUTPUT_PATH]
[--server-stop-command SERVER_STOP_COMMAND] [--startup-skip-identify]
[--startup-wait STARTUP_WAIT_SECS]
RUN_SERVER_COMMAND
Setup a directory as if it is being graded. This is useful for seeing what the autograder will see right before grading begins.
usage: python3 -m autograder.cli.testing.setup-grading-dir [-h] [--version]
[--testing-mode] -a
ASSIGNMENT -s
SUBMISSION
[-o OUT_DIR] [-ss]
Submit multiple assignments to an autograder and ensure the output is as expected.
usage: python3 -m autograder.cli.testing.test-remote-submissions
[-h] [--version] [--server SERVER] [--user USER] [--pass PASS]
[--course COURSE] [--assignment ASSIGNMENT] [--message MESSAGE]
[--allow-late] [--testing-mode]
SUBMISSIONS_DIR
Run a grader against multiple test assignments and ensure the output matches the expected output.
usage: python3 -m autograder.cli.testing.test-submissions [-h] [--version]
[--testing-mode]
[-a ASSIGNMENT] -s
SUBMISSIONS
Verify test data by starting the specified server and testing HTTP exchanges against it.
usage: python3 -m autograder.cli.testing.verify-test-data [-h] [--version]
[--testing-mode]
[--server-output-file SERVER_OUTPUT_PATH]
[--server-stop-command SERVER_STOP_COMMAND]
[--startup-skip-identify]
[--startup-wait STARTUP_WAIT_SECS]
RUN_SERVER_COMMAND
TEST_DATA_DIR
The `autograder.cli.users` package contains tools to manage autograder users.
Authenticate as a user.
usage: python3 -m autograder.cli.users.auth [-h] [--version] [--server SERVER]
[--user USER] [--pass PASS]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
Get information about a server user.
usage: python3 -m autograder.cli.users.get [-h] [--version] [--server SERVER]
[--user USER] [--pass PASS]
[--target-email TARGET_EMAIL]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers] [--skip-headers]
List the users on the server.
usage: python3 -m autograder.cli.users.list [-h] [--version] [--server SERVER]
[--user USER] [--pass PASS]
[--target-users TARGET_USERS]
[--testing-mode]
[--format {json,table,text}]
[--include-extra-fields]
[--pretty-headers]
[--skip-headers]
The `autograder.cli.users.password` package contains tools to manage user passwords.
Change your password to the one provided.
usage: python3 -m autograder.cli.users.password.change [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
--new-pass NEW_PASS
[--testing-mode]
Request a password reset.
usage: python3 -m autograder.cli.users.password.reset [-h] [--version]
[--server SERVER]
[--user USER]
[--testing-mode]
Remove a user from the server.
usage: python3 -m autograder.cli.users.remove [-h] [--version]
[--server SERVER] [--user USER]
[--pass PASS]
[--target-email TARGET_EMAIL]
[--testing-mode]
The `autograder.cli.users.tokens` package contains tools to manage user tokens.
Create a new token.
usage: python3 -m autograder.cli.users.tokens.create [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--target-user TARGET_USER]
[--name NAME]
[--testing-mode]
Delete a token.
usage: python3 -m autograder.cli.users.tokens.delete [-h] [--version]
[--server SERVER]
[--user USER]
[--pass PASS]
[--target-user TARGET_USER]
--token-id TOKEN_ID
[--testing-mode]
List tokens for a user.
usage: python3 -m autograder.cli.users.tokens.list [-h] [--version]
[--server SERVER]
[--user USER] [--pass PASS]
[--target-user TARGET_USER]
[--testing-mode]
Upsert server users from a TSV file.
usage: python3 -m autograder.cli.users.upsert-file [-h] [--version]
[--server SERVER]
[--user USER] [--pass PASS]
[--dry-run]
[--skip-inserts]
[--skip-updates]
[--send-emails]
[--testing-mode]
PATH
Upsert a server user.
usage: python3 -m autograder.cli.users.upsert [-h] [--version]
[--server SERVER] [--user USER]
[--pass PASS] [--dry-run]
[--skip-inserts]
[--skip-updates] [--send-emails]
--new-email NEW_EMAIL
[--new-name NEW_NAME]
[--new-pass NEW_PASS]
[--new-role {unknown,user,creator,admin,owner}]
[--new-course NEW_COURSE]
[--new-course-role {unknown,other,student,grader,admin,owner}]
[--new-lms-id NEW_LMS_ID]
[--testing-mode]
The `autograder.cli.util` package contains general utility commands.
Pull the code from a file and output the sanitized version as if it was being graded.
usage: python3 -m autograder.cli.util.extract-code [-h] [--version]
[--testing-mode]
[-o OUT_PATH]
PATH
Check the style of all '.py' and '.ipynb' files in the paths specified (recursively).
usage: python3 -m autograder.cli.util.style [-h] [--version] [--testing-mode]
[--ignore-path IGNORE_PATHS]
[--ignore-pattern IGNORE_PATTERNS]
[--style-overrides STYLE_OVERRIDES]
path [path ...]