Script Testing Framework

Data Structures
class	Test
	Attribute used for tests annotation and assignment to Suites. More...
class	TestBase
	Test base class. More...
class	TestHarness
	Collection and main interface of the Testing framework. More...

Functions
TestBase Managed	Failure ()
	Base class for test results. This way you report back to the system. More complex failure types with descriptions can be reported by implementation of FailureText in format of junit [https://llg.cubic.org/docs/junit/].
string	FailureText ()
	Text used for xml report output.
proto string	GetName ()
	Suite class name getter. Strictly for UI porposes!
proto native int	GetNTests ()
	Returns the number for tests within this suite.
proto native TestBase	GetTest (int handle)
	Returns a test.
proto native bool	IsEnabled ()
	Enabled flag getter.
proto native bool	NativeFailure ()
proto native string	NativeFailureText ()
void	OnInit ()
	Callback for user defined initialization. Called for all suites during TestHarness.Begin().
proto native void	SetEnabled (bool val)
	Enables/Disables the suites. Disabled suites won't run at all.
TestHarness Managed	SetResult (TestResultBase res)
	Collection of tests.
void	Step (EStage stage=EStage.Main)

Variables
class Test	Main
class Test	Setup
	Stage definition used in conjunction with Step attribute.
class Test	Stage
	Attribute which marks a method as part of the testing process.

Detailed Description

Script Testing Framework

Introduction

• Provides a unified and simple interface that emphasizes the smallest amount of boiler plate code possible.
• The collection and instantiation of its primitives is performed right after the script compilation.
• Within the framework a SINGLE test harness derived class can exist. The harness runs the tests and contains API to access them.
• The test units are compiled to Suites. These provide additional API for environmental control.

Simple tests

Can be perfomed in form of annotated free functions.

Note: Notice the Suite name in the attribute.

[Test("MyFirstTestSuite")]
TestResultBase MyFooBarTest() { return TestBoolResult(5 > 3); }

Stateful tests

More elaborate tests that need some state and will run for several ticks have to be defined as TestBase derived classes. Your logic has to be ran through step methods.

Step methods

• You can name your step methods however you like.
• They have to be annotated by the [Step(Stage)] attribute which pairs the step with a stage.

Stages

• They divide the steps into groups that express the initialization and finalization process.
• Stages are executed in order Setup -> Main -> TearDown.
• Methods in stages are executed in order of definition.

Return values

• void -> Will get executed only once.
• bool -> Will get executed every tick until true is returned.

Result

• Result is set via API member method of TestBase SetResult(TestResultBase).
• Setting a result which evaluates to failure will terminate the stage.

Failure unwind

• If the Setup stage fails the test only terminates and TearDown is not called.
• Main stage failure will trigger the TearDown.
• TearDown failure will do nothing.

Timeout

• The tests won't timeout by default. The value may be specified via Test attribute.
• The timeout counter resets for every stage method.
• If the stage method times out the TimeoutResult is set (it evaluates to failure and the failure unwind process starts).

[Test("MyFirstTestSuite", timeoutS: 2, timeoutMs: 250)]
class MyAsyncTest : TestBase
{
    // Simple blocking initialization.
    [Step(EStage.Setup)]
    void Initialize() { ... }
 
    // Async test which is waiting for result for several frames.
    [Step(EStage.Main)]
    bool Pool() { ... }
 
    // Finalization process waiting for result for several frames.
    [Step(EStage.TearDown)]
    bool FinalizeA() { ... }
 
    // Simple blocking finalization call.
    [Step(EStage.TearDown)]
    void FinalizeB() { ... }
}

Function Documentation

Failure()

TestBase Managed Failure ( )

protected

Base class for test results. This way you report back to the system. More complex failure types with descriptions can be reported by implementation of FailureText in format of junit [https://llg.cubic.org/docs/junit/].

Return true of the result means failure.

Definition at line 176 of file testingframework.c.

FailureText()

string FailureText ( )

protected

Text used for xml report output.

Definition at line 206 of file testingframework.c.

References NativeFailureText().

GetName()

proto string GetName

(

)

Suite class name getter. Strictly for UI porposes!

Test name getter. Strictly for UI porposes!

Definition at line 50 of file syncedvalue.c.

References m_Name.

GetNTests()

proto native int GetNTests

(

)

Returns the number for tests within this suite.

GetTest()

proto native TestBase GetTest

(

int

handle

)

Returns a test.

IsEnabled()

proto native bool IsEnabled

(

)

Enabled flag getter.

Referenced by PointLightBase::OnFrameLightSource(), and SpotLightBase::PerformVisibilityCheck().

NativeFailure()

proto native bool NativeFailure ( )

protected

NativeFailureText()

proto native string NativeFailureText ( )

protected

Referenced by FailureText().

OnInit()

override void OnInit ( )

protected

Callback for user defined initialization. Called for all suites during TestHarness.Begin().

Definition at line 81 of file freezestate.c.

References GetPlugin(), PluginBase::InitWidgets(), m_ConfigDebugProfile, m_DebugRootWidget, m_Duration, m_ID, m_SymptomType, and vector::Zero.

Referenced by AnimationTimer::AnimationTimer(), TimerBase::Timer(), and TimerBase::WidgetFadeTimer().

SetEnabled()

proto native void SetEnabled

(

bool

val

)

Enables/Disables the suites. Disabled suites won't run at all.

Enables/Disables the test. Disabled tests won't run at all.

SetResult()

proto native void SetResult

(

TestResultBase

res

)

Collection of tests.

Sets the test result. Failure can result in specialized behavior described in TestResultBase.

Sets the suite result. Failure can result in specialized behavior described in TestResultBase.

References Stage.

Step()

void Step

(

EStage

stage = EStage.Main

)

Definition at line 129 of file testingframework.c.

Variable Documentation

Main

class Test Main

Setup

class Test Setup

Stage definition used in conjunction with Step attribute.

Referenced by CAContinuousQuantityRepeat::Execute(), CAContinuousTimeCooking::Execute(), and CABase::Init().

Stage

class Test Stage

Attribute which marks a method as part of the testing process.

Referenced by SetResult().