When developing software, a significant part of the production process relies on software testing. What this is and how such activities are carried out will be discussed in this article.
What is called testing?
By this we mean the process during which the software is executed in order to detect places of incorrect functioning of the code. To achieve the best result, difficult sets of input data are intentionally constructed. The main objective of the inspector is to create optimal opportunities for software product failure . Although sometimes testing a developed program can be simplified to the usual test of performance and performance of functions. This saves time, but is often accompanied by software unreliability, user dissatisfaction, and so on.
Efficiency
How well and quickly errors are found, significantly affects the cost and duration of software development of the required quality. So, despite the fact that testers receive wages several times lower than programmers, the cost of their services usually reaches 30 - 40% of the cost of the entire project. This is due to the number of personnel, since it is an unusual and rather difficult process to look for a mistake. But even if the software passed a substantial amount of tests, there is no 100% guarantee that there will be no errors. It is simply not known when they will appear. To motivate testers to choose the types of checks that are more likely to find a mistake, various means of motivation are used: both moral and material.
Approach to work
The optimal situation is when various mechanisms are implemented to ensure that there are no errors in the software from the very beginning. To do this, it is necessary to take care of competent architecture design, a clear technical task, and it is also important not to make corrections in connection when work on the project has already begun. In this case, the tester is faced with the task of finding and determining a small number of errors that remain in the final result. This will save both time and money.
What is a test?
This is an important aspect of the activity of the verifier, which is necessary for the successful detection of shortcomings in the program code. They are necessary in order to control the correctness of the application. What is included in the test? It consists of initial data and values, which should turn out as resulting (or intermediate). In order to more successfully identify problems and inconsistencies, tests need to be compiled after the algorithm has been developed, but programming has not begun. Moreover, it is desirable to use several approaches in calculating the necessary data. In this case, the probability of detecting an error increases due to the fact that you can examine the code from a different point of view. Integrated tests should provide verification of the external effects of the finished software product, as well as its operation algorithms. Of particular interest are marginal and degenerate cases. So, in practice, activities with errors can often be revealed that the cycle operates one time less or more than it was planned. It is also important to test the computer, thanks to which you can check the compliance with the desired result on various machines. This is necessary in order to make sure that the software can work on all computers. In addition, testing the computer on which development will be carried out is important when creating multi-platform development.
The art of finding mistakes
Programs are often aimed at working with a huge array of data. Is it really necessary to create it completely? Not. The practice of "miniaturization" of the program has become widespread. In this case, there is a reasonable reduction in the amount of data compared to what should be used. Let's look at an example: there is a program in which a 50x50 matrix is created. In other words, you must manually enter 2500 thousand values. This, of course, is possible, but it will take a very long time. But to test the operability, the software product receives a matrix whose dimension is 5x5. To do this, you will need to enter 25 values. If in this case a normal, error-free operation is observed, then this means that everything is in order. Although there are pitfalls here, which are that during miniaturization a situation occurs, as a result of which changes become implicit and temporarily disappear. It is also very rare, but still it happens that new errors appear.
Pursued goals
Software testing is not an easy thing because this process cannot be fully formalized. Larger programs almost never have the necessary exact benchmark. Therefore, a number of indirect data is used as a guideline, which, however, cannot fully reflect the characteristics and functions of software developments, which are being debugged. Moreover, they must be selected so that the correct result is calculated even before the software product is tested. If this is not done in advance, then there is a temptation to consider everything approximately, and if the machine result falls into the expected range, an erroneous decision will be made that everything is correct.
Testing under various conditions
As a rule, testing of programs takes place in volumes that are necessary for a minimal check of functionality within a limited range. Activities are carried out with changing parameters, as well as their working conditions. The testing process can be divided into three stages:
- Verification under normal conditions. In this case, the main functionality of the developed software is tested. The result should be as expected.
- Check in emergency conditions. In these cases, it is assumed to obtain boundary data that can adversely affect the performance of the created software. An example is the work with extremely large or small numbers, or in general, the complete absence of information received.
- Exception Check. It involves the use of data that lies beyond processing. In such situations, it is very bad when the software perceives them as suitable for calculation and produces a plausible result. Care must be taken to ensure that in such cases there is a rejection of any data that cannot be correctly processed. It is also necessary to provide for informing the user about this.
Software Testing: Types
Creating software without errors is very difficult. This takes a considerable amount of time. To get a good product, two types of testing are often used: Alpha and Beta. What are they? When talking about alpha testing, then it means a test conducted by the staff of developers in "laboratory" conditions. This is the last stage of the test before the program will be transferred to end users. Therefore, developers are trying to maximize. For ease of operation, data can be logged to create a chronology of problems and their solutions. Beta testing means the delivery of software to a limited circle of users so that they can run the program and identify missing errors. The peculiarity in this case is that often the software is not used for its intended purpose. Thanks to this, malfunctions will be detected where nothing has been noticed before. This is quite normal and there is no need to worry about it.
Test completion
If the previous steps were successfully completed, then it remains to conduct an acceptance test. In this case, it becomes a simple formality. During this check, confirmation is made that no additional problems were found and the software can be released to the market. The more important the final result will be, the more carefully the verification should be carried out. It is necessary to ensure that all stages are completed successfully. This is how the testing process as a whole looks. Now, let's dig deeper into technical details and talk about useful tools like test programs. What are they and in what cases are they used?
Automated Testing
Previously, it was believed that the dynamic analysis of the developed software was too heavy an approach that is inefficient to use for detecting defects. But because of the increasing complexity and volume of the programs, the opposite view appeared. Automated testing is used where performance and safety are the most important priorities. And they should be with any input. As an example of programs for which such testing is appropriate, the following can be cited: network protocols, web server, sandboxing. We will further examine several samples that can be used for such an activity. If you are interested in free testing programs, then among them it’s quite difficult to find quality ones. But there are hacked "pirated" versions of well-proven projects, so you can turn to their services.
Avalanche
This tool helps to detect defects by testing programs in dynamic analysis mode. It collects data and analyzes the execution path of the developed object. The tester is provided with a set of input data that cause an error or bypass the set of existing restrictions. Due to the presence of a good verification algorithm, a large number of possible situations are developed. The program receives various sets of input data that allow you to simulate a significant number of situations and create such conditions when a failure is most likely. An important advantage of the program is the use of heuristic metrics. If there is a problem, then the application error is highly likely. But this program has limitations like checking only one marked input socket or file. During an operation such as program testing, detailed information will be provided on the presence of problems with null pointers, infinite loops, incorrect addresses or malfunctions due to the use of libraries. Of course, this is not a complete list of detected errors, but only their common examples. Alas, developers will have to correct the flaws - automatic tools are not suitable for these purposes.
KLEE
This is a good memory testing program. It can intercept approximately 50 system calls and a large number of virtual processes, thus running in parallel and separately. But in general, the program does not look for individual suspicious places, but processes the maximum possible amount of code and analyzes the data transmission paths used. Because of this, the testing time of the program depends on the size of the object. When checking, the bet is placed on symbolic processes. They are one of the possible ways to complete tasks in a program that is being tested. Thanks to parallel work, a large number of options for the application under study can be analyzed. For each path, after the end of its testing, the input data sets from which the verification began are saved. It should be noted that testing programs with KLEE helps to identify a large number of deviations, which should not be. She can find problems even in applications that have been developed for decades.