Guide for Automated Test Best Practices | Part 3 Optimize Test System

Part 3 | Test System Integration, Deployment and Sustainability

This is part 3 of our three-part series on automated test best practices. For best results, check out part 1 The Three Most Important Things in Test System Design" and part 2 "The Enjoyment of Deployment" before reading part 3 below.

Our third “C” word is the biggest communication framework of all. You already have design engineering and production in test engineering’s communication framework. Believe it or not, we need to bring sales, marketing, and all your test vendors into the picture! You see, sales and marketing cannot promote the product until the factory has produced enough products to start distribution. Nothing happens until you have the test specification signed off, test system defined, and test vendors who can help you choose the proper switching, instrumentation, cabling, etc. Production has allocated space and operators for the test system(s) necessary, and only then can the production manager push the start button on the assembly process!  

Checklist for Test System Integration and Deployment  

1. Test specifications and requisite tests that require mandatory sign-off.
2. Define quantity and specifications of stimulus and measurement channels (voltage, current, frequency, power, etc.). 
3. Have product launch time agreed to by all stakeholders, including design, test, production, sales and marketing.
4. Design test system - evaluate and contact test vendors for chassis, switching, power, measurement, cabling, etc., to review requirements, make recommendations and place orders.
5. Formalize anticipated throughput (number of products tested per shift, per work week, etc.)
6. Analyze average test time (including load/unload times) divided by daily throughput equals number of testers needed.
   • This parameter could affect the orders generated in #4.
   • Is there adequate floor space for the test department?
7. Integrate and program test systems
   • Are there adequate test engineering staff?
   • If not, should you hire new or contract a third party?
8. Deploy test system(s) and train operators.
9. Relax! A job well done. 

Actually, if you did not think about long-term sustainment of the test systems, you can’t relax yet. In Part 2, we talked about testing to minimize errors in tests to avoid lawsuits and lost wars. Regarding the latter (defense products), you have another issue to address, long-term support of the systems, or...  

Obsolescence Management 

The critical words here are long-term sustainability. Most defense products need to be supported for a long, long, long time. In the commercial world, 1 to 5 years is typical. But we are talking 10, 15 years or more in the defense industry! In the world of electronics, that seems like several lifetimes. But, if you visit a military base, you might see test systems from the 1990s based on old standards like GPIB or VXI. These systems have to support the military for an extended period. And so do your test systems. 

So, what should I be doing, or what should I have done, in this instance?

Questions to Ask of Your Vendors and Yourself. 

• Do you use proactive planning to manage obsolescence risks, i.e., work closely and regularly with your suppliers to understand the life-cycle status of all the component parts and S/W used in your test systems?
• What are your vendors' policies for long-term availability?  
• If a module becomes obsolete, do they guarantee to provide a new replacement with the same form, fit and function? 
• If not, are you confident they will work with you to provide a suitable alternative solution that can be implemented with minimum disruption?
• Alternatively, do you need to look for an alternative vendor or a third-party company that can provide this ongoing support?
• Do you also need to consider stocking spares and/or a "last-time buy" of potentially at-risk products?
•  Are you able to schedule incremental upgrades to your test systems over time? Scheduled maintenance can provide opportunities to insert new technologies or add new features.

• If a company or companies providing chassis, switching and instrumentation goes out of business, or the modular standard you chose early in the design phase becomes obsolete, are there other modular standards that could replace your present systems in a pinch? 

• In this instance, how easy will it be to port your software over to the new test system and/or modular standard? 
• How close to the original test system modules are the planned replacements in terms of specifications? 

• Cabling and test cabinets should not be an issue, as there are always other vendors to supply these parts. However, in the case of cabling, if you do not have well documented wiring diagrams, how do you build replacement cables when one fails on an existing system? If you are building cables in-house, you probably have adequate documentation. But if you use a third party, ensure they provide sufficient documentation and diagrams in case they go out of business.

Test System Sustainability

While your strategy to sustain your automated test system needs to be concerned with “health”, remember that switching is in the front line of your test system and is exposed to faulty units under test (UUTs) and programming errors, especially during debug. These factors can accidentally subject the relays to signals that exceed their specification, resulting in stuck open, welded contacts or high resistance connections. These faults must be diagnosed and repaired as quickly as possible to maximize production throughput. Test system self-test routines can often only discover that the switching system as a whole has a fault, and it must then be returned to the manufacturer for repair, a process that can take several weeks.

To eliminate this repair cycle, some switch vendors make proprietary self-test capabilities available, which can quickly diagnose individual faulty relays—the customer’s technicians can easily repair these, and the test system can be up and running again in a matter of hours. The following video gives an example of how Pickering helps customers maximize the up-time & optimize their test systems.

But test system sustainability is not just about diagnostics but also a bit of prognostics. Test system components like test fixtures, mass interconnects, cables, and even air filters all need servicing and replacing. A detailed support plan helps assure sustainability.

Earlier in this post, I mentioned talking to test hardware and software vendors to know how long they can support your test requirements. Where possible, get commitment in writing from the vendors. That may not help if they go out of business or get acquired, but that salesperson may feel more obligated to warn you as early as legally possible.  After all, this person may go to a different test vendor and will try to sell to your department again!

Finally, I can’t emphasize enough that a big part of sustainability that is the responsibility of test engineering… is documentation. Accurate information on design, instrumentation, calibration periods, wiring diagrams, system updates and more will allow you to easily maintain your test systems going forward and build more as necessary. Too often, I have seen a test engineering department where “old reliable Joe” designed that last test system, and when he retired, you discover that the design was only in Joe’s head! ☹

Conclusion 

So, to conclude, you can see that a successful test implementation takes many departments and vendors and strong communications so that all members are aware of the milestones and work together toward eventual success. 

So, start as early as possible, develop a strong communication framework, meet regularly to review progress, collaborate with vendors who understand the test requirements and act like part of the test engineering team, select the right products while keeping in mind long-term support, get the proper switching and simulation products in a timely fashion, integrate the right test system for your application and document everything. 

Link to Part 1 of this blog series.

Link to Part 2 of this blog series.

Automated Test