Think Strategically for Design of Experiments Success (3)

Experiment Administration and Data Collecection

Administration of an experimental program is a vital step that is often overlooked. It must be carefully planned and executed. As John Wooden, arguably the most successful college basketball coach ever, has admonished us, “Failing to plan is planning to fail.”

The first aspect of planning is to make sure that you have all the personnel, equipment, and materials needed to construct the experimental program. There should be no waiting around, which can be a big source of waste in R&D laboratories.

An experiment should be randomized and conducted in the randomized order. If randomization isn’t done or runs are derandomized to make an experimental work more convenient, then a critical aspect of experimentation is ignored.

Another important issue is finding out during testing that experimental combinations couldn’t be completed. To help prevent this, evaluate all runs in a design for operability. If some are suspect, then make those runs first to see whether they can be done. If not, consider changing their levels (e.g., move the run closer to the center of the design) or changing the range of critical variables and recreating the design. Several options will become clear in each situation. The critical point is to consider the situation up front and plan for how to deal with an eventuality should it occur.

Data Analysis and Modedel Building

Many points could be made here; I will address a few. First you need a process for conducting analysis such as that shown in the “Method for Building Process and Product Models” box (14). This roadmap gives you a plan to follow and points out key events that must happen.

Next, you need to make effective use of graphics, a best practice for using statistical thinking and methods. Graphics work because of humans’ ability to see patterns in data that are too complex to be detected by statistical models.

When a poor fit of a model is obtained (a low adjusted R2 value), it can be a result of several factors, including an important variable missing from the model, the wrong model being used (e.g., a linear model used when the response function is curved), and atypical values in the data (outliers). These issues are fairly well known. Poor measurement quality can be a source of poor fit of a model to the data. Your model may be correct and the fit have a low adjusted R2 value as a result of high measurement variation introduced by a poor measurement system.

Residual analysis is another aspect of good statistical practice. A residual is the difference between the observed measurement and the value predicted by the model Y = f(X). Residual analysis provides much information (e.g., the presence of missing variables, outliers, and atypical values; and a need for curvature terms in a model).

Confirmation Studieses

A fundamental of good experimental practice is completing confirmation experiments. This documents that the results and recommendation of your experiments can be duplicated and that the model you developed for the system accurately predicts response behavior. Confirmation experiments are best conducted in an environment where results will be used such as the manufacturing process instead of a laboratory.

Findings, Conclusions, and Rececommendations

It is good practice to make oral presentations of findings and recommendations before writing a report (14). Present first to a small group of stakeholders to assess reaction and receptivity to your results, then present to broader audiences. You can improve your presentation using input you receive from various groups. Afterward, you are in a position to write a report (that will probably go unnoticed because your results will already be accepted and thus be old news).
Presentation and written reports should contain graphics summarizing and communicating important findings. Graphs might include histograms, dot plots, main effects, two-factor interactions, and cube plots. A good graphic is understood by both presenter and user and is simple and easy to understand. There is elegance in simplicity.

Be careful when reproducing computer outputs directly into presentations and reports. Most computer outputs contain more information than your audience needs. Reduce displays of computer output to only what is needed to communicate your message. It is acceptable to put more detailed computer output in an appendix, but you need to be careful to make sure that a reader will need all the information presented.
Effecective Usese of DoE Createses A Competitive Advantage
Using DoE enables effective use of QbD in creating product and process design spaces and process control strategies and supports risk management. Like any other tool, its elements, strengths, and limitations must be understood for DoE to be used effectively. The “Tips and Traps” box provides a list of factors that enable effective use of QbD. Those suggestions were developed over years of successful projects and were effective in a number of different situations.
The promise of effective DoE is that the route of product and process development will speed up through more cost-effective experimentation, product improvement, and process optimization. Your “batting average” will increase, and you will develop a competitive advantage in the process.

Referencescesces
1 Snee RD. Quality by Design — Four Years and Three Myths Later. Pharm. Process. February 2009: 14–16. www.pharmpro.com/articles/2010/03/government-and-regulatory-Implementing-Quality-by-Design.
2 Snee RD. Building a Framework for Quality by Design. Pharm. Technol. online, October 2009, http://pharmtech.findpharma.com/pharmtech/Special+Section%3a+Quality+by+Design/Building-a-Framework-for-Quality-by-Design/ArticleStandard/Article/detail/632988?ref=25.
3 Snee RD. Robust Strategies for Improving Upstream Productivity. BioPharm Int. 23(6) 2010: S28–S33.
4 ICH Q8. Harmonized Tripartite Guideline: Pharmaceutical Development. Current Step 4 Version. International Conference on Harmonisation of Technical Requirements for the Registration of Pharmaceuticals for Human Use: Geneva, Switzerland, 10 November 2005.
5 Juran JM. Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services. The Free Press, New York, NY 1992.
6 Lonardo A, Snee RD, and Qi B. Time Value of Information in Design of Downstream Purification Processes — Getting the Right Data in the Right Amount at the Right Time. BioPharm Intl. 23(3) 2010: 23–28.
7 Snee RD. Raising Your Batting Average: Remember the Importance of Strategy in Experimentation. Quality Progress December 2009: 64-68.
8 Yan L, Le-he M. Optimization of Fermentation Conditions for P450 BM-3 Monooxygenase Production by Hybrid Design Methodology. J. Zhejian University Science B 8(1): 27–32.
9 Hulbert MH, et al. Risk Management in Pharmaceutical Product Development: White Paper Prepared by the PhRMA Drug Product Technology Group. J. Pharm. Innovation 3, 2008: 227–248.
10 Schweitzer M, et al. Implications and Opportunities of Applying QbD Principles to Analytical Measurements. Pharm. Technol. 33 (2) 2010, 52–59
11 Box GEP, Hunter JS, and Hunter WG. Statistics for Experimenters — Design, Innovation, and Discovery. Wiley-Interscience: New York, NY, 2005.
12 Montgomery DA. Design and Analysis of Experiments, 7th Edition. John Wiley and Sons: New York, NY, 2005.
13 Snee RD. Computer-Aided Design of Experiments: Some Practical Experiences. J. Quality Technol. 17(4) 1985; 222–236
14 Hoerl RW, Snee RD. Statistical Thinking: Improving Business Performance, Duxbury Press: Pacific Grove, CA, 2002.

About the author:
Ronald D. Snee, PhD is founder and president of Snee Associates, a firm dedicated to the successful implementation of process and
organizational improvement initiatives. He provides guidance to senior executives in their pursuit of improved business performance
using QbD, Lean Six Sigma, and other improvement approaches. Ron received his BA from Washington and Jefferson College and MS
and PhD degrees from Rutgers University. He is a frequent speaker and has published four books and more than 200 papers.