Hi again,
Following on on your specific patch, I've done a little bit more
thinking on how we might want to use Avocado for more extensive
testing.
I think I mentioned during some of our calls my idea of having some
sort of test side library to automatically turn test code into
executables that can be used from Avocado's extended exec stuff. I
don't think I explained it very clearly though.
I've made a super rudimentary draft of this idea here:
https://gitlab.com/dgibson/exeter
The "example.py" program, run without arguments lists all the ways you
can run it, each of which constitutes a single test case.
The idea is that that "list" mode would be extended to generate the
json manifest that Avocado can consume. So first you run it to
generate the json, then you feed that into avocado which will run each
of the cases as a separate test. Ideally avocado could do both steps
itself (this is what I was talking about with the "variant 2" of the
extended exec stuff). But we can do it externally for the time being.
Having things separated this way makes it super easy to manually
re-run a specific test case. We can also allow the same library to
emit the list of tests in other formats for other runners (being able
to do this for meson seems like it might be pretty useful).
Obviously, it would need a bunch more functionality to be really
useful: ways of automatically generating matrices of tests, composing
functions together to make test cases, as well as basic helper
functions like assert variants.
--
David Gibson (he or they) | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you, not the other way
| around.
http://www.ozlabs.org/~dgibson