Measurement/How to Succeed

Loading...

How to Succeed

Tips for Teams

Before you begin working in the lab, you should think about how you will measure your results. Careful measurement practices are a hallmark of successful iGEM projects (see exemplary projects). But think about going beyond validating your own project: the most impressive teams produce robust, reproducible results that others can build on for years to come.

Here are a few general tips to get you started:

• Be creative! We love seeing new and innovative approaches that showcases what’s unique about your measurement activities.
• Understand the limits of your methods. No instrument or assay is perfect. Learn the range of signals that can be measured, the precision that can be expected, and the typicals types of errors and artifacts. Determine how many replicates are necessary, include process controls, and report an appropriate number of significant figures.
• Communicate clearly about units and controls. Report in a way that shows the source and validation of your data so that the judges and teams better understand what you have accomplished.
• Use the measurement kit provided as part of the iGEM distribution and other suggested resources
• Provide data that others can use. Consider what you can do to help other teams reuse and apply your work. How might your approach be used in different contexts?

In addition, here are four specific rules to follow to ensure good measurement practices:

1. Report measurements in standard, comparable units. Do not use arbitrary or relative units. The resources page gives methods for calibrating fluorescence data.
2. Always include process controls to validate your protocol and instruments. Process controls are samples with known behavior, such as fluorescein, wild-type cells, and GFP driven by a strong constitutive promoter. These should give the same behavior in every experiment. If they do not, then you know there is likely a problem with your experimental data.
3. Use appropriate statistics. Gene expression is a complex catalytic reaction, and for that reason its variation is expected to be multiplicative rather than additive, and we should generally compute geometric statistics instead: means and standard deviations on the logarithm of the data. It is also important to distinguish between the mean standard deviation, such as you'd want to use to report the variation in single cell behavior, and the standard deviation of means, such as you'd want to use to report the amount of variation between replicates.
4. Present data clearly. Write your units on your axes, include process control information for comparison, distinguish between geometric and arithmetic statistics, and use a log axis when presenting geometric statistics. Visualize your data in a way that shows as much of your data as possible, instead of using methods like bar graphs that hide the underlying data (see figure below).

Part of Figure 1 from Weissberger et al 2015, "Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm". When possible, it is best to plot your data in such a way that all of the raw data can be seen. For example, using a bar graph with error bars can be misleading because it obscures the raw data-- all of the univariate scatterplots in (B-E) could generate the bar graph shown in (A).

Quantitative vs. Qualitative Measurements

An important part of designing an experiment is deciding what to measure and how. This section describes the two main types of measurements, quantitative and qualitative, and provides examples of how to use them.

Quantitative Measurements are reported with numerical values. Ideally they will be reported in units that have a physical meaning. In general, you should try and make quantitative measurements whenever possible, particularly if you want other researchers in the future to compare their results to yours.
When making quantitative measurements, you should make sure that:

• You follow the four measurement tips described above: report measurements in standard units, include controls, use appropriate statistics, and present data clearly.
• You should also provide the raw values of your data in a table or spreadsheet.

Qualitative Measurements are typically descriptive, as they measure categorical variables and so do not involve numerical values. Categorical variables have a fixed set of values (such as “true or false” or “low, medium, and high”) which must be defined by the experimenter. Qualitative measurements can also include relative results, such as whether one condition is “more” or “less” than another condition.
When making qualitative measurements, you should make sure that:

• You describe your definitions for your categories clearly and thoroughly. Sometimes this might involve using quantitative values, such as for thresholds between categories. These definitions should be presented alongside your data.
• Your categories are defined unambiguously so that measurements will not fall into more than one category for one variable. For example, if your categories are “low”, “medium”, and “high”, they should be defined so that one measurement cannot be both “medium” and “high” for the same variable.

Examples of Quantitative and Qualitative Measurements

Below are two examples that display good measurement practices for teams designing their experiments.

Example 1 (Quantitative Measurement):
A team is measuring GFP fluorescence in a plate reader. Using the fluorescein standards from the Measurement Kit, the team converts their measurements from arbitrary units to absolute units (molecules of equivalent fluorescein). This way, their results can be compared directly to results from other labs that are reported in the same units.
Once the team obtains their data, they plot their results in a way that shows the important features of the data clearly, with replicates shown. They also post the raw data that they obtained from the plate reader as a spreadsheet on a file-sharing service and provide a link to the data on their wiki. This way, future researchers will be able to incorporate this data into their own analyses.

Example 2 (Qualitative Measurement):
A team is conducting a color-based staining assay on bacterial colonies growing on agar plates to determine the presence of a polymer. Although the team would like to perform a quantitative measurement, they do not have any specialized equipment that would be able to measure numerical values of color.
The team is considering taking pictures of the plates and then calculating the pixel intensities for each colony and reporting the result, hoping that colored colonies will have different intensities than non-colored colonies. However, in this situation it is best for the team to perform a qualitative measurement.
They should create a uniform color standard to compare their colonies against, and use the standard as a threshold to determine if the colonies are colored or not. If the team tried to use pixel intensity as a quantitative value, then they risk introducing extra factors such as lighting conditions in different pictures into their data. As long as the team describes their reference standards and documents this information alongside their results, the qualitative measurement will be better-suited for the situation.