changelog shortlog graph tags branches changeset files file revisions raw help

Mercurial > core / annotate lisp/lib/rt/pkg.lisp

changeset 698: 96958d3eb5b0
parent: 3dd1924ad5ea
author: Richard Westhaver <ellis@rwest.io>
date: Fri, 04 Oct 2024 22:04:59 -0400
permissions: -rw-r--r--
description: fixes
632
bbd9024f2fe2 lib/rt upgrades and refactoring
Richard Westhaver <ellis@rwest.io>
parents: 582
diff changeset
1
 ;;; pkg.lisp --- regression testing packages
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
2
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
3
 ;; Regression Testing framework. inspired by PCL, the original CMUCL
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
4
 ;; code, and the SBCL port.
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
5
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
6
 ;;; Commentary:
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
7
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
8
 ;; - :rt https://www.merl.com/publications/docs/TR91-04.pdf Chapter 1
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
9
 ;; - :com.gigamonkeys.test https://github.com/gigamonkey/monkeylib-test-framework
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
10
 ;; - :sb-rt https://github.com/sbcl/sbcl/blob/master/contrib/sb-rt/rt.lisp
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
11
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
12
 ;; This package is intended to provide a modernized Lisp testing
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
13
 ;; library with features found in some of the test frameworks listed
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
14
 ;; below.
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
15
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
16
 ;; - :it.bese.fiveam https://github.com/lispci/fiveam
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
17
 ;; - :try https://github.com/melisgl/try
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
18
 ;; - :rove https://github.com/fukamachi/rove
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
19
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
20
 ;;; TODO:
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
21
 #|
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
22
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
23
 - [ ] benchmark support: do-bench, test-count, 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
24
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
25
 - [ ] fixtures api
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
26
 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
27
 - [ ] profiling 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
28
 |#
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
29
 ;;; Code:
449
7ca2e98f0847 x updates
Richard Westhaver <ellis@rwest.io>
parents: 448
diff changeset
30
 (in-package :std-user)
108
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
31
 (defpackage :rt
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
32
   (:use 
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
33
    :cl :std :sxp :log
286
237756e1358b enforcing log level (todo), rt finessing to get x to test instead of fail
Richard Westhaver <ellis@rwest.io>
parents: 284
diff changeset
34
    :sb-aprof)
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
35
   (:export
260
b9cd591b9c10 io prims
Richard Westhaver <ellis@rwest.io>
parents: 229
diff changeset
36
    :test-error
110
cae8da4b1415 rocksdb macrofication, fixes for RDB (C Strings will be the death of me), removed PWN - quicklisp package conflict, smh.
ellis <ellis@rwest.io>
parents: 109
diff changeset
37
    :*test-opts*
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
38
    :*compile-tests*
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
39
    :*catch-test-errors*
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
40
    :*test-suffix*
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
41
    :*default-test-suite-name*
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
42
    :*test-suite*
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
43
    :*test-suite-list*
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
44
    ;;  TODO 2023-09-04: :*test-profiler-list* not yet
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
45
    :*testing*
513
4cdf71621bae more tpc-h
Richard Westhaver <ellis@rwest.io>
parents: 503
diff changeset
46
    :random-elt
4cdf71621bae more tpc-h
Richard Westhaver <ellis@rwest.io>
parents: 503
diff changeset
47
    :random-ref
4cdf71621bae more tpc-h
Richard Westhaver <ellis@rwest.io>
parents: 503
diff changeset
48
    :random-char
4cdf71621bae more tpc-h
Richard Westhaver <ellis@rwest.io>
parents: 503
diff changeset
49
    :random-chars
4cdf71621bae more tpc-h
Richard Westhaver <ellis@rwest.io>
parents: 503
diff changeset
50
    :random-bytes
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
51
    :test-suite-designator
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
52
    :check-suite-designator
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
53
    :make-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
54
    :make-suite
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
55
    :test-name=
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
56
    :do-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
57
    :do-tests
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
58
    :reset-tests
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
59
    :continue-testing
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
60
    :with-test-env
108
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
61
    :%test-bail
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
62
    :%test-result
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
63
    :make-test-result
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
64
    :ensure-suite
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
65
    :test-fixture
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
66
    :fixture-prototype
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
67
    :make-fixture-prototype
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
68
    :make-fixture
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
69
    :with-fixture
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
70
    :test-result
108
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
71
    :test-fn
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
72
    :test-pass-p
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
73
    :test-fail-p
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
74
    :test-skip-p
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
75
    :test-failed
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
76
    :fail!
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
77
    :is
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
78
    :signals
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
79
    :deftest
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
80
    :defsuite
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
81
    :in-suite
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
82
    :eval-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
83
    :compile-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
84
    :locked-tests
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
85
    :push-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
86
    :pop-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
87
    :delete-test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
88
    :find-test
282
da580c7fe954 upgrades
Richard Westhaver <ellis@rwest.io>
parents: 274
diff changeset
89
    :find-suite
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
90
    :do-suite
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
91
    :test-object
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
92
    :test
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
93
    :test-fixture
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
94
    :test-suite
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
95
    :test-name
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
96
    :tests
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
97
    :test-form
632
bbd9024f2fe2 lib/rt upgrades and refactoring
Richard Westhaver <ellis@rwest.io>
parents: 582
diff changeset
98
    :test-results
bbd9024f2fe2 lib/rt upgrades and refactoring
Richard Westhaver <ellis@rwest.io>
parents: 582
diff changeset
99
    :*tmp*
bbd9024f2fe2 lib/rt upgrades and refactoring
Richard Westhaver <ellis@rwest.io>
parents: 582
diff changeset
100
    :*default-tmp-directory*
bbd9024f2fe2 lib/rt upgrades and refactoring
Richard Westhaver <ellis@rwest.io>
parents: 582
diff changeset
101
    :with-tmp-directory
bbd9024f2fe2 lib/rt upgrades and refactoring
Richard Westhaver <ellis@rwest.io>
parents: 582
diff changeset
102
    :with-tmp-file))
96
301fd45bbe73 big refactor of lisp code
ellis <ellis@rwest.io>
parents:
diff changeset
103
 
108
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
104
 (defpackage :rt/bench
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
105
   (:nicknames :bench)
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
106
   (:use :cl :std :log :rt)
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
107
   (:export
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
108
    :*bench-count*
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
109
    :defbench
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
110
    :do-bench))
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
111
 
260
b9cd591b9c10 io prims
Richard Westhaver <ellis@rwest.io>
parents: 229
diff changeset
112
 (uiop:define-package :rt/cover
110
cae8da4b1415 rocksdb macrofication, fixes for RDB (C Strings will be the death of me), removed PWN - quicklisp package conflict, smh.
ellis <ellis@rwest.io>
parents: 109
diff changeset
113
   (:nicknames :cover)
448
a37b1d3371fc fasl tweaks
Richard Westhaver <ellis@rwest.io>
parents: 437
diff changeset
114
   (:use :cl :std :log :rt)
282
da580c7fe954 upgrades
Richard Westhaver <ellis@rwest.io>
parents: 274
diff changeset
115
   (:export
da580c7fe954 upgrades
Richard Westhaver <ellis@rwest.io>
parents: 274
diff changeset
116
    :with-coverage :start-coverage :stop-coverage
da580c7fe954 upgrades
Richard Westhaver <ellis@rwest.io>
parents: 274
diff changeset
117
    :*coverage-directory*
286
237756e1358b enforcing log level (todo), rt finessing to get x to test instead of fail
Richard Westhaver <ellis@rwest.io>
parents: 284
diff changeset
118
    :coverage-report))
110
cae8da4b1415 rocksdb macrofication, fixes for RDB (C Strings will be the death of me), removed PWN - quicklisp package conflict, smh.
ellis <ellis@rwest.io>
parents: 109
diff changeset
119
 
215
0d11384aae81 rt tests
Richard Westhaver <ellis@rwest.io>
parents: 211
diff changeset
120
 (defpackage :rt/tracing
0d11384aae81 rt tests
Richard Westhaver <ellis@rwest.io>
parents: 211
diff changeset
121
   (:nicknames :tracing)
108
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
122
   (:use :cl :std :log :rt)
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
123
   (:export
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
124
    :start-tracing
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
125
    :stop-tracing
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
126
    :with-tracing
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
127
    :save-report
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
128
    ;; Extra utility
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
129
    :package-symbols-except))
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
130
 
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
131
 (defpackage :rt/flamegraph
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
132
   (:nicknames :flamegraph)
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
133
   (:use :cl :std :log :rt :sb-sprof)
215
0d11384aae81 rt tests
Richard Westhaver <ellis@rwest.io>
parents: 211
diff changeset
134
   (:export :save-flamegraph))
108
ac7ef5ddcaab rt/bench rt/flamegraph rt/trace
ellis <ellis@rwest.io>
parents: 96
diff changeset
135
 
577
806c2b214df8 refactored q/tests, added rt/fuzz, more sql query work
Richard Westhaver <ellis@rwest.io>
parents: 514
diff changeset
136
 (defpackage :rt/fuzz
806c2b214df8 refactored q/tests, added rt/fuzz, more sql query work
Richard Westhaver <ellis@rwest.io>
parents: 514
diff changeset
137
   (:nicknames :fuzz)
806c2b214df8 refactored q/tests, added rt/fuzz, more sql query work
Richard Westhaver <ellis@rwest.io>
parents: 514
diff changeset
138
   (:use :cl :std :log :rt)
806c2b214df8 refactored q/tests, added rt/fuzz, more sql query work
Richard Westhaver <ellis@rwest.io>
parents: 514
diff changeset
139
   (:export :fuzzer
806c2b214df8 refactored q/tests, added rt/fuzz, more sql query work
Richard Westhaver <ellis@rwest.io>
parents: 514
diff changeset
140
            :fuzz
582
568c39371122 sql updates, fuzzz
Richard Westhaver <ellis@rwest.io>
parents: 577
diff changeset
141
            :fuzz*
568c39371122 sql updates, fuzzz
Richard Westhaver <ellis@rwest.io>
parents: 577
diff changeset
142
            :fuzz-generator
568c39371122 sql updates, fuzzz
Richard Westhaver <ellis@rwest.io>
parents: 577
diff changeset
143
            :fuzz-state))