7 Testing
7.1 Test Suites
This section describes the public tests suites provided by the library. See test-suite, rackunit, run-tests, and rackunit/text-ui.
Largely syntactic unit tests for each milestone. These are based on the milestones, not the chapter, and are fragile. They make assumptions about the structures of pass list, and can break if students use a different structures. They are deprecrated and exist for backwards compatibility.
Property-based test suites for each chapter. These are based on the language versions and are much more robust, as they test via the language interpreters and validators. These make no assumptions about the structure of the pass list. They are strongly recommended.
The suites work best when the compiler is designed and implementated top-down, and may not test intermediate passes until earlier passes are complete, as the testing framework generates tests for later passes from earlier passes that pass their own tests.
7.1.1 Version 1 Test Suites
| (require cpsc411/test-suite/public/v1) | |
| package: cpsc411-lib | |
procedure
(v1-public-test-suite pass-list interp-list check-paren-x64 interp-paren-x64) → test-suite? pass-list : (listof (-> any/c any/c)) interp-list : (listof (or/c #f (-> any/c any/c))) check-paren-x64 : (-> any/c paren-x64-v1?) interp-paren-x64 : (-> paren-x64-v1? (in-range 0 255))
pass-list is expected to be a list of passes that compile from paren-x64-v1? to x64, compatible with current-pass-list.
interp-list must be the same length as pass-list, and should be a list of interpreters for the source language of the corresponding pass in pass-list, or #f to indicate this pass has no interpreter and cannot be independently tested. Such passes will be composed together and tested against the next available interpreter. The test suite uses execute with an empty pass list as the final target language interpreter.
For example, given a pass list (list check-paren-x64 generate-x64 wrap-x64-run-time wrap-x64-boilerplate) and interpreter list (list interp-paren-x64 interp-paren-x64 #f #f), check-paren-x64 will be tested by interpretering both the input and output with interp-paren-x64. The passes generate-x64, wrap-x64-run-time, and wrap-x64-boilerplate are composed together, and tested by interpreting source programs with interp-paren-x64, and target programs through execution.
7.1.2 Version 2 Test Suites
| (require cpsc411/test-suite/public/v2) | |
| package: cpsc411-lib | |
procedure
(v2-public-test-suite pass-list interp-list) → test-suite? pass-list : (listof (-> any/c any/c)) interp-list : (listof (or/c #f (-> any/c any/c)))
Reuses all test suites from v1-public-test-suite where possible.
pass-list is expected to be a list of passes that compile from asm-lang-v2? to x64, compatible with current-pass-list.
See v1-public-test-suite for details about the interpretation of pass-list and interp-list.
7.1.3 Version 3 Test Suites
| (require cpsc411/test-suite/public/v3) | |
| package: cpsc411-lib | |
procedure
(v3-public-test-suite pass-list interp-list) → test-suite? pass-list : (listof (-> any/c any/c)) interp-list : (listof (or/c #f (-> any/c any/c)))
pass-list is expected to be a list of passes that compile from values-lang-v3? to x64, compatible with current-pass-list.
See v1-public-test-suite for details about the interpretation of pass-list and interp-list.
7.1.4 Milestone 1 Test Suites
| (require cpsc411/test-suite/public/a1) | |
| package: cpsc411-lib | |
procedure
(a1-public-test-suite pass-list interp) → test-suite?
pass-list : (listof (-> any/c any/c)) interp : (-> paren-x64? integer?)
7.1.5 Milestone 2 Test Suites
| (require cpsc411/test-suite/public/a2) | |
| package: cpsc411-lib | |
procedure
(a2-public-test-suite pass-list uncover-locals assign-fvars replace-locations check-paren-x64 interp-values-lang interp-paren-x64) → test-suite? pass-list : (listof (-> any/c any/c)) uncover-locals : (-> asm-lang-v2? asm-lang-v2/locals?) assign-fvars : (-> asm-lang-v2/locals? asm-lang-v2/assignments?) replace-locations : (-> asm-lang-v2/assignments? nested-asm-lang-v2?) check-paren-x64 : (-> any/c paren-x64-v2?) interp-values-lang : (-> values-lang-v3? int64?) interp-paren-x64 : (-> paren-x64-v2? int64?)
7.2 Test Lang
The language adds the reader macro $, which behaves like , (unquote), but calls the current-input-encoder. to encode a Racket value into a valid value for the CPSC411 compiler. For much of the compiler, this is simply the identity function: Racket numbers in the int64 range are also CPSC411 values. Once we add tagged datatypes, this encoding changes.
You can use cpsc411/test-lang only as a #lang.
#lang cpsc411/test-lang `(begin (set! rax 1)) `(begin (set! rax $1)) '(begin (set! rax $1)) (parameterize ([current-input-encoder (lambda (x) (* x 8))]) `(begin (set! rax $1)))
Reads the same as:
> (require cpsc411/test-suite/utils) > `(begin (set! rax 1)) '(begin (set! rax 1))
> `(begin (set! rax ,((current-input-encoder) 1))) '(begin (set! rax 1))
> '(begin (set! rax ,((current-input-encoder) 1))) '(begin (set! rax ,((current-input-encoder) 1)))
> (parameterize ([current-input-encoder (lambda (x) (* x 8))]) `(begin (set! rax ,((current-input-encoder) 1)))) '(begin (set! rax 8))
7.3 Test Suite Utils
| (require cpsc411/test-suite/utils) | package: cpsc411-lib |
parameter
(current-input-encoder) → (-> any/c any/c)
(current-input-encoder encoder) → void? encoder : (-> any/c any/c)
= values
parameter
(current-expected-masker) → (-> any/c any/c)
(current-expected-masker masker) → void? masker : (-> any/c any/c)
= values
procedure
(exit-code-mask v) → (between/c 0 255)
v : any/c