How To Write UI Tests

From KDevelop

1 How to compile and run tests
2 UI Testing Examples
- 2.1 Simple Shell test from scratch
- 2.2 Simple Sublime test from scratch
3 UI Testing Techniques

[edit] How to compile and run tests

Just configure kdevplatform with

 cmake -DKDE4_BUILD_TESTS=on

and you're all set.

To run all tests use

 make test

command from toplevel dir or from sublime/ and shell/ subdirs.

To run individual test just run its executable:

 cd sublime/tests
 ./sublime-viewactivationtest

To run individual test function just pass the name of the function as the parameter:

 cd sublime/tests
 ./sublime-viewactivationtest testActivationAfterViewRemoval

You can pass several test functions there. Also there are more options that tests accept (verbosity and output control, etc.). See http://doc.trolltech.com/4.4/qtestlib-manual.html#qtestlib-command-line-arguments for the reference or just run test with --help from the command line.

[edit] UI Testing Examples

[edit] Simple Shell test from scratch

[edit] shellexampletest.h

#include <QObject>
#include <tests/common/autotestshell.h>
using namespace KDevelop;

class ShellExampleTest: public QObject {
    Q_OBJECT
private slots:
    void init();
    void cleanup();
    void testSomething();
};

[edit] shellexampletest.cpp

#include "shellexampletest.h"
#include <tests/common/kdevtest.h>
#include <QtTest/QtTest>

void ShellExampleTest::init() {
    AutoTestShell::init();  // create instance of test shell extension
    KDevelop::Core::initialize(); // initialize kdevplatform
}
void ShellExampleTest::cleanup() {
}
void ShellExampleTest::testSomething() {
    // your test here
    // expect full blown kdevplatform-based application 
    // loaded and initialized by this moment (albeit without any plugins because we currently don't have them for test shell)
}
//THIS IS VERY IMPORTANT: use KDEVTEST_MAIN macro for ALL UI tests
KDEVTEST_MAIN(ShellExampleTest)
#include "shellexampletest.moc"

[edit] CMakeLists.txt

set( EXECUTABLE_OUTPUT_PATH ${CMAKE_CURRENT_BINARY_DIR} )
include_directories(${CMAKE_SOURCE_DIR} ${CMAKE_SOURCE_DIR}/interfaces ${CMAKE_SOURCE_DIR}/shell)

set( shellexampletest_SRCS shellexampletest.cpp )
kde4_add_unit_test(shell-exampletest ${shellexampletest_SRCS})
target_link_libraries(shell-exampletest ${QT_QTTEST_LIBRARY} ${KDE4_KDEUI_LIBS} kdevplatformshell )

[edit] Simple Sublime test from scratch

I won't post an example here because Sublime tests are just usual tests (but also created with KDEVTEST_MAIN macro). Some Sublime tests launch simple application, some of them just create several objects and test their functionality. Take a look at existing Sublime tests because 99% of the time you'll just add new test functions to them.

[edit] UI Testing Techniques

Writing UI tests is actually very easy and fun to do. Here are some useful UI test writing techniques.

[edit] Test mostly user experience instead of internals

The main idea behind UI tests we write for Sublime and Shell is to replicate user actions and verify results that user would get. This way of thinking greatly simplifies tests, reduces the time you spend writing the test and actually assures that UI is never broken for the user.

Some guidelines:

call high-level public functions that are directly called by the user interface and check their results, for example

   IDocumentController *documentController = Core::self()->documentController();
   documentController->openDocumentFromText("");
   QCOMPARE(documentController->openDocuments().count(), 1);

find QAction objects in the UI and trigger them manually, for example:

   QAction *splitAction = Core::self()->uiControllerInternal()->activeMainWindow()->actionCollection()->action("split_vertical");
   QVERIFY(splitAction);
   splitAction->trigger();
   QCOMPARE(doc->views().count(), 2);

simulate events, for example:

   qApp->sendEvent(myView->widget(), new QFocusEvent(QEvent::FocusIn));
   QCOMPARE(mainWindow->activeView(), myView);

try to check signals you emit, especially if the signal is the part of public API (lots of plugins will connect to the signals and will expect them to work reliably), for example:

   QSignalSpy spy(controller, SIGNAL(viewAdded(Sublime::View*)));
   area->addView(doc->createView());
   QCOMPARE(spy.count(), 1);

[edit] Aggregate test results before making comparisons and verifications

With multiple QVERIFY and QCOMPARE test functions may become messy. It is sometimes a good idea to print some data into string and compare that string to the original data also encoded in the string.

For an example, see sublime/tests/areaoperationtest.cpp. We print the layout of mainwindow's central area to the string and compare the string with expected layout:

checkAreaViewsDisplay(mainWindow, area, QString("\n\
[ vertical splitter ]\n\
    [ vertical splitter ]\n\
        [ view2.1.1 view2.1.2 ]\n\
        [ view2.4.1 ]\n\
    [ horizontal splitter ]\n\
        [ view2.5.1 ]\n\
        [ view2.3.1 ]\n\
"));

This check assures that after we show the particular area in some mainwindow we get several splitters created and those splitters contain some visible views inside.

One such check replaces 12-15 QCOMPARE()'s and QVERIFY()'s and makes test look easier to understand because it's immediately obvious for the reader that mainwindow should have vertical splitters with two splitters inside. Both of internal splitters should contain some views, etc, etc.

[edit] Write less tests and more test functions

Each test class is linked to the separate executable. Linking too many executables may be really slow. Also each Sublime test is a full-featured KDE application and takes some time to start. Each Shell test is not only a KDE application but a KDevPlatform application so it starts even slower.

So the morale is, do not spawn too many test executables without the real need. It is always possible to run just several test functions from the test in case you don't want to wait for the whole test to pass:

#cd sublime/tests
#./sublime-controllertest --functions
testDocumentDeletion()
testAreaDeletion()
testNamedAreas()
#./sublime-controllertest testNamedAreas testAreaDeletion
...

[edit] Keep test code clean

As I've said before, the goal of writing UI tests is to specify certain behavior we expect to work. Of course that specification should be succinct and easily understandable for the reader. Please expect that your test will be read by other developers. When they break your code, they will start looking from the test, not from the code.

[edit] Feel good ;)

Once you write the test for your feature or bugfix, you can feel safe and warm ;) Your work will never be broken/lost/etc. because there's now an automated way to assure that.

Retrieved from "http://www.kdevelop.org/mediawiki/index.php/How_To_Write_UI_Tests"

This page has been accessed 2,908 times. This page was last modified 11:40, 3 May 2008. Content is available under GNU Free Documentation License 1.2.