<< Previous chapter (Testing API) | Next chapter (Functions & checkers) >>

Testing results

This section describes what are testing results and how to interpret certain situations and sql codes. All testing procedures will continue run tests for specified function till first failure will occur. If that failure won't happen, then all tests were passed and success result will be returned.

So, testing result may be:

• Success test passing. That means, all tests ended with expected result. In this case, no error trace will be returned and server will send successful testing procedure call code to the client.
• Failure on some test. Then, in general, result will consist of two parts:
  • Failure testing procedure call state with the corresponding SQL Exception code will be sent to the client
  • If call was made in verbose mode (for example, via *_VERBOSE call for front-API), error trace will be included as a row set.

If error occured, then SQL Exception code will always be like 8****, which is specific for mysql-unit. Each code has it's meaning and may be used in external libraries to determine error reason.

Trace for failure

Testing trace will always contain same fields, but some of them may be NULL, depending of logical meaning of that field in terms of error, which had happened.

Full fields list is:

• test_id
  Identifier of test, which caused failure. If it was procedure testing and failure was derived from some check in TEST_PROCEDURE_RESULTS, then this field will be set to id of that check.
• test_expression
  Expression, which caused failure. Usually, it is the field from the corresponding tests table, but in case of procedures testing there is an exception from that rule: if procedure call itself thrown mismatch between test expectation and actual result (for example, test expected that error will happen during procedure call and that error had not happened), then this field will contain procedure call string (so, procedure name together with arguments list from TEST_PROCEDURE_ARGUMENTS table)
• test_value
  A value for expression, that was expected in test
• test_is_error
  A flag which indicates either test is expected to cause error or not.
• test_error_code_expected
  Error code, which is expected in test
• test_error_code_thrown
  Error code, which was thrown by test run or NULL is not error happened
• test_error_message_thrown
  Message for error, which was thrown by test run or NULL is not error happened
• test_trace
  Literal representation of trace. This is a human-readable compilation of previous testing trace fields.

If all tests were successful, then testing trace won't appear. Also, testing trace is always represented with one and only one row set, which contains exactly one row.

SQL Codes for failure

Below is a list of SQL codes for exceptions, which are used in error state, which is sent to client, when testing procedure call faced a failure in some test:

• Code: 80000
  Message: Test id < X > : assertion failed
  Explanation: There is a mismatch between expected result of test expression evaluation and it's actual value. This error has lowest priority and will happen only if match check for error throwing for test was passed.
• Code: 80100
  Message: Tests for specified entity name or test number were not found
  Explanation: No tests were found for specified entity (procedure or function) name. If main API was used and test id was passed, then this error means that test with that id was not found in the corresponding tests storage table.
• Code: 80200
  Message: Test id < X > : expectation of Y failed.
  Explanation: Expectation for Y was failed for test with id X. Here Y can be "ERROR" or "VALIDITY". This error means that there is a mismatch between error expectation and actual result. Thus, either error happened while it wasn't expected, or it wasn't thrown while it is expected in the test.
• Code: 80300
  Message: Test id < X > : expectation of specific ERROR CODE failed.
  Explanation: Test X is expected to throw error, and that was happened, but the actual error code differs from error code, which is expected in test.
• Code: 80400
  Message: Test id < X > : expectation of Y failed.
  Explanation: Same as state 80200, but for check in TEST_PROCEDURE_RESULTS table.
• Code: 80500
  Message: Specified DDL unit is not yet supported
  Explanation: This is reserved SQL state code, but it is used only for built-in DDL checkers. It won't appear as error code, which is sent to client. Instead this code is a regular SQL state code, which may be used as expected code inside tests. Meaning of this code is: there is no DDL checker for specified DDL unit type.
• Code: 80600
  Message: Environment integrity check failed
  Explanation: This code means, that some of session-specific variables, which are used inside testing procedures, are not NULL at the moment, when testing procedure was invoked. That means, your current connection is using some of those variables, and, due to security reasons, test procedure execution was interrupted. This is a way to prevent mysql-unit from overwriting user session data. However, if this error happened, then you should reconsider naming of your session variables in order to continue using mysql-unit.
• Code: 80700
  Message: X meta-field "Y" does not exist
  Explanation: This is reserved SQL state code, but it is used only for built-in DDL getters. It won't appear as error code, which is sent to client. Instead this code is a regular SQL state code, which may be used as expected code inside tests. Meaning of this code is: there is no DDL "Y" field for specified DDL "X" unit, derived from invoked DDL getter.

Adding custom SQL codes

You are free to add codes with 8**** mask, but they should not conflict with reserved SQL codes. To make it safe, there is a reserved block for user-defined codes, it's 88***. Codes within that block will never used inside mysql-unit in future versions and can be safely used for user-defined SQL exceptions

User-defined SQL codes are like code 80500 - so, they have specific prefix only to show, that they belong to mysql-unit scope, but they are regular codes and may be used, for example, inside tests as expected error codes. The reserved 80*** codes can not be used there, since they have specific flow-control meaning.