--- /dev/null
+Unit Testing for Diogenes
+===========================
+
+Unit testing is the process of writing lots of little pieces of test code
+which each test some part of the functionality of a software system. By
+having a comprehensive set of test cases which are satisfied by the
+production code, we can have real confidence in our software, instead of
+just "hoping" it works.
+
+What is a unit test?
+----------------------
+
+The general structure of all unit tests is as follows:
+
+* Setup pre-conditions (the state of the system prior to the execution of the
+ code under test)
+* Run the code to be tested, giving it all applicable arguments
+* Validate that the code ran correctly by examining the state of the system
+ after the code under test has run, and ensuring that it has done
+ what it should have done.
+
+The pre-conditions are normally either set at the beginning of the test
+method, or in a general method called setUp() (see below for the structure
+of test code).
+
+Running the code itself is normally making a call to the function to
+be tested, constructing an object, or, for Web applications, running the
+top-level web script (using require()).
+
+Validating the post-run state of the system is done by examining the
+database, system files, and script output, and using various assert methods
+to communicate the success or failure of various tests to the user.
+
+Testing Web applications such as Diogenes is relatively easy, especially the
+user interface side of things.
+
+Unlike traditional GUI applications, every state change of a web application
+is defined by the database and filesystem, the user's session, and the
+values passed from the web browser through $_REQUEST and $_SERVER. Since
+all of these aspects are relatively easy to control, setting the pre-run
+state of the application is quite simple. There are also relatively few
+discrete states that your web application can be in, because all of the
+system control has to be made through the constrained interface above.
+
+Testing post-conditions, as well, is simple. If you want to ensure that a
+certain thing is part of the post-run display, you can just interrogate the
+HTML output from the test run, which is all text and can be tested with
+assertRegExp() and assertSubStr(). Again, the only state available is in
+the user's session, database, and filesystem, all of which are easy to
+interrogate.
+
+Test cases and test methods
+-----------------------------
+
+(This is where things get a bit hairy -- hold on, and perhaps read it twice)
+
+A test case is a collection of tests which are related in some way. Each
+test case is represented in our testing framework by a class subclassed from
+PHPUnit_TestCase. Each test is a method on the test case whose name starts
+with 'test'. You should name the tests appropriately from there.
+
+Test cases can have a couple of special methods defined on them, called
+setUp() and tearDown(). The first method is called before each of the test
+methods is called, and the second is called after each test method has run.
+
+The setUp() and tearDown() methods are the primary reason for grouping test
+methods together. Methods should, as much as possible, grouped into test
+cases with common setUp() and tearDown() requirements.
+
+So how do I write a Unit Test?
+-------------------------------
+
+Put together a snippet of code which sets up the state of the application,
+then run the code to be tested, and check the result. This snippet of code
+should be placed in a test case in a method named test[Something](). Each
+test method takes no arguments and returns nothing.
+
+Have a look at the existing test code in the testing/ directory of the
+diogenes source distribution for examples of how to write test code.
+
+When you create a new test case, you need to tell the test code runner that
+it needs to run the new test code. There is an array in the alltests script
+which defines the files and test cases to be run. Each key in the array
+specifies a filename to be read (without the .php extension), while the
+value associated with the key is an array of class names in that file.
+
+What should I test?
+---------------------
+
+The naive answer would be "everything". However, that is impractical.
+There are just too many things that could be tested for a policy of "test
+everything" to allow any actual code to be written.
+
+It helps to think of tests as pulls of the handle on a poker machine. Each
+pull costs you something (some time to write it). You "win" when a test
+that you expected to pass fails, or when a test you expected to fail passes.
+You lose when a test gives you no additional useful feedback. You want to
+maximise your winnings.
+
+So, write tests that demonstrate something new and different about the
+operation of the system. Before you write any production code, write a test
+which defines how you want the system to act in order to pass the test. Run
+the test suite, verify that the test fails. Now, modify the production code
+just enough so the test passes. If you want the system to do something that
+can't be expressed in one test, write multiple tests, each one interspersed
+with some production development to satisfy *just* *that* new test. This
+is, in my experience, the best way to ensure that you have good test
+coverage, whilst minimising the production of tests which add no value to
+the system.
+
+How do I retro-fit unit testing onto an existing codebase?
+------------------------------------------------------------
+
+Diogenes already has a significant amount of code written, which would take
+hundreds of tests to cover completely. There is little point in going back
+and writing tests for all of this functionality. It appears to work well
+enough, so we should just leave it as-is.
+
+However, from now on, every time you want to make some modification (whether
+it be a refactoring, a bug fix, or a feature addition), write one or more
+test cases which demonstrate your desired result:
+
+Refactoring: Write tests surrounding the functionality you intend to
+ refactor. Show the test cases accurately represent the desired
+ functionality of the system by ensuring they all run properly. Then
+ perform the refactoring, ensuring you haven't broken anything by
+ making sure the tests all still run properly.
+
+Bug fix: Write one or more tests which shows the bug in action -- in other
+ words, it hits the bug, and produces erroneous results. Then modify
+ the system so that the test passes. You can be confident that
+ you've fixed the bug, because you have concrete feedback in the form
+ of your test suite that the bug no longer exists.
+
+Feature addition: Put together some tests which invoke the feature you want
+ to add, and test that the feature is working as it should.
+ Naturally, these tests will fail at first, because the feature
+ doesn't exist. But you then modify the production code to make the
+ feature work, and you stop when your tests all pass.
+
+Over time, as old code gets modified, the percentage of code covered by the
+tests will increase, and eventually we will have a comprehensively tested
+piece of software.
+
+During modifications, if you manage to break something accidentally, write a
+test to show the breakage and fix it from there. If you broke it once,
+there's a good chance it'll break again when someone else modifies it, and
+there should be a test to immediately warn the programmer that they've
+broken something.
+
+How do I run the unit tests?
+------------------------------
+
+The primary script that executes all of the unit tests is the 'alltests'
+script in the testing/ directory of the distribution. However, the output
+of this script is one line for every test that passes or fails.
+
+To help highlight the test failures, there is a 'run' script, which filters
+out all of the passes, and only shows you the failures. Very useful.
+
+So, your regular test run will be done with ./run, but if you want to see a
+thousand passes, run ./alltests. Both of these should be run from the
+testing/ directory. Running it from elsewhere isn't likely to get good
+results.
+
projects / diogenes.git / blobdiff