TESTING AUTOMATION
Test and Evaluation Approach
Requirements decomposition drives:
- Component Test Cases
- Integration Test Threads
- Functional Test Threads
Agile development process with continuous feedback from the test results. Test cases and test threads captured in a Test Management System.
- Provide traceability to requirements and development activities
- Capture details on test setup, procedures, and pass/fail criteria
- Can be linked to specific test events
- TMS example includes Redcase for Redmine
I&T Continuous Integration
Continuous integration (CI) tools used to automate builds and testing:
- Provide rapid feedback to developers based on latest changes
- Regression testing to maintain existing functionality
- Tracking of test results over time
- Reduces risk of technology insertion
- Verifies component functionality before integration
DAS experience with CI Tools:
Jenkins CI, Bamboo, Sikuli GUI automation, Selenium Web Driver
I&T Orchestration Tools
Orchestration Tools Capabilities
- Maintains proper test configuration and operation
- Enables consistent and well documented test environment
- Ensures test repeat-ability
- Properly collects all test artifacts
Test Configuration
- Chef, Puppet, Salt's tools available
- Automate system configuration and update
- Provides means for strong configuration via version control
Test Execution
- Sikuli and Selenium can be used to drive GUI and web applications
- EASE is capable of orchestrating component configuration, startup, shutdown, and artifact collection
- Particularly useful for Integration Events
Incremental Test Events
Integration Events and Functional Verification
- Incrementally test integration of system
- Isolated component test cases for entry in IE
- Orchestration tools to set up consistent testing
- Test automation incorporated
System Measurement Performance
- Capture system performance metrics under load to identify bottlenecks
- Utilize system agents and network statistics
- Cisco NetFlow and Performance Monitor
- Apache ActiveMQ statistics