Cross-sectional studies: Longer duration of participant recruitment does not make them longitudinal

Sometimes researchers think that they are doing a longitudinal study simply because their study requires a longer duration to recruit the needed number of study participants. However, it is not the length of time it takes to enroll the needed number of study participants that defines cross-sectional studies. It is the temporal relationship of the ascertainment of exposure and outcome that matters.

In cross-sectional studies exposure and outcome are ascertained at the same time. That is both are measured at the same point in time. One doesn’t precede the other. Measurement per study participant is done only once. No repeat/follow up measurement at some later time.

Conversely, the distinguishing feature of longitudinal studies is repeated measurement. Two or more measurements are done per study participant at given time intervals. The temporal relationship between exposure and outcome is such that the outcome comes later than the exposure. 

To conclude, a cross-sectional study may take a year to accomplish whereas a longitudinal study may be accomplished in few weeks or months. They can simply be distinguished by the fact that the former involves a single measurement per study participant (no repeat/follow up measurement at a later time) and ascertainment of exposure and outcome is done at the same time while the latter involves at least one baseline measurement and at least one other follow up measurement and the presumed exposure is ascertained before the outcome occurs.  

Ethics review of public health studies

Without any need to go historically back to the Nazi human experimentation or the Tuskegee Syphilis Study or so to justify the need for research ethics, now it is a standard practice that all public health (and other) studies involving humans or human biological materials shall be conducted as per accepted ethical principles. To ensure the ethical soundness of such studies, it is a requirement that study protocols undergo a rigorous ethics review by research ethics committees (RECs) or institutional review boards (IRBs).

 But what part of the research protocols shall undergo review by RECs or IRBs? Some researchers are not happy when RECs or IRBs review parts of the protocol other than the ethics statement section. Shall RECs or IRBs review only the ethics statement of public health research protocols?

RECs or IRBs shall review a protocol in its entirety, not just the ethics statement section. Anything from research problem identification to the design of the study methodology and to the data analysis, interpretation and reporting of findings could raise ethical issues. It is, therefore, crucial that public health research protocols be reviewed in their entirety (from title to annexes). Generally, review of a public health research protocol shall address the following: 1) scientific review, 2) conflict of interest review and 3) ethics review.

The three types of reviews are briefly discussed below.

 Scientific review. Scientific review is required to ensure the scientific soundness of the proposed study. This includes reviewing whether the study has a clear research question and objectives, whether it is going to generate new knowledge, whether the methodology is appropriate and so on.  A study that is not scientifically sound cannot be ethical. For example, a study that is not likely to come up with new knowledge, and that is likely to “reinvent the wheel”, unnecessarily bothers study participants and unnecessarily wastes meager resources and is hence unethical.

 Conflict of interest review. This is required to ensure that parties that are directly or indirectly involved in the research do not have a vested interest that influences the researchers. Such influences may included, for example, influence of a funding agency on how the research is to be conducted, whether results should be published, where and when to publish the findings and cherry-picking of findings for publication. A review of review articles on the health effects of passive smoking published between 1980 and 1995 by Barnes and Bero^[1] has found out that review articles published by tobacco industry-affiliated authors had about 88 times higher odds of concluding that passive smoking has no health effect compared to authors who are non tobacco-industry affiliated, which shows the influence of funding agencies on authors. Hence, conducting public health studies which have a conflict of interest is unethical and hence RECs or IRBs shall ensure that proposed studies are free from such conflicts of interest.

Ethics review. This is required to ensure that the proposed study be conducted in accordance with the three basic principles, viz. respect for persons, beneficence and justice.^[2]  A study that violates any of these basic principles is unethical.

To conclude, ethics review of public health research protocols shall not be restricted only to the ethics statement section of the protocols. It should rather review the protocols in their entirety and make sure that planned studies are scientifically sound, free from conflict of interest and comply with the basic ethical principles of studies involving humans or human biological materials. 

  

Reference

1.      Barnes DE, Bero LA. Why Review Articles on the Health Effects of Passive Smoking Reach Different Conclusions. JAMA. 1998;279(19):1566-70.

2.      Council for International Organizations of Medical Sciences (CIOMS). International Ethical Guidelines for Biomedical Research Involving Human Subjects. Geneva; 2002.

Do not control a variable belonging to a sub-sample

In public health research, multivariable models are often used to control confounders while investigating the effect of an exposure on an outcome. To use a given model, certain assumptions shall be fulfilled. A model cannot be valid if its assumptions are fatally violated. Some assumptions are checked before using the model. For example, one wouldn’t choose a linear regression model for a nominal dependent variable because the assumption of “a continuous dependent variable” is violated. Other assumptions are checked post-hoc (for example, normality of residuals in linear regression models). Assumptions are model-specific.

There is also a prerequisite that must be checked for all models but that is not often taught to students and not clearly addressed in books. A variable being controlled in a multivariable model should be one that has been measured on all study units and hence for which all study units have some meaningful value. A control variable should not be one that is measured only on a sub-sample. For example, in a study among mothers, if there is a variable about outcome of previous pregnancy, it applies only to mothers who had at least one pregnancy in the past. It is not uncommon to find researchers who enter such variables into multivariable models and end up in models “behaving wildly”. This is because all study units for which the variable doesn’t apply will be excluded from the analysis. The analysis will then be limited to a sub-sample. In effect, estimates will biased, precision will be lost as manifested by too wide confidence intervals (sometimes, the lower and upper bounds of confidence intervals could not be estimated), and model goodness-of-fit statistics cannot be determined.

          

A variable belonging to a sub-sample can be controlled only if one is purposely doing the analysis on a sub-sample for a sub-population inference. In that case, it should be well planned from the outset including ensuring the adequacy of the sample size for sub-population inference.

In conclusion, while working with multivariable models, it s necessary to make sure that all control variables (also exposure variables, for that matter) apply for all study units (unless one is doing an analysis of a sub-sample for sub-population inference). Otherwise, results could lack both validity and precision.

Collider-stratification bias

In public health research, adjusting for confounders using multivariable methods is a popular practice. That is necessary to deal with the menace of confounders. However, adjusting for confounders should be accomplished with the utmost care because a not-well-thought-about adjustment can take the data “from the frying pan into the fire”. To this end, multivariable analysis should not be a laissez faire practice where one plugs in as many variables as possible into the multivariable model and then see what comes out of the model. It should rather be carefully guided by a conceptual framework or causal diagrams. The mechanism by which each variable operates should be duly considered and analysis done accordingly. Otherwise, an unwise attempt to control so-called confounders would end up in biasing estimates.

      

Among such biases is a collider-stratification bias. Collider is “a variable directly affected by two or more variables in a causal diagram”^[1] (Figure 1). If variable C is directly affected by both variables X and Y, stratifying on (or controlling) C while assessing the association of X with Y, or the association of both X and Y with some other variable Z, tends to change the association between the variables X and Y^{[1, 2]}. That is, it may cause a spurious association to emerge between X and Y while there is none^{[2, 3]}; it may also cause a statistical association between the outcome of interest and an unmeasured factor U that is also causal of C while actually the variables are independent^[4] thus forcing the unmeasured variable U to be a confounder. Controlling the collider may also cause a risk factor to appear protective^[4]. Such biases resulting from controlling a collider variable during data analysis are called collider-stratification biases^[2] or biases due to conditioning on a collider^[3].

Fig. 1: A collider variable. Variable C is directly affected by both variables X and Y.

Ipso facto, as a way of avoiding collider-stratification bias, collider variables should not be controlled (or adjusted for) in multivariable models ^[2].

References and additional reading materials:

1.       Porta M, editor. A Dictionary of Epidemiology. 5th ed. New York: Oxford University Press; 2008.

2.       Greenland S. Quantifying Biases in Causal Models: Classical Confounding vs Collider-Stratification Bias. Epidemiology. 2003;14(3):300 –6.

3.       Cole SR, Platt RW, Schisterman EF, Chu H, Westreich D, Richardson D, et al. Illustrating bias due to conditioning on a collider. International Journal of Epidemiology. 2010;39:417-20.

4.       Whitcomb BW, Schisterman EF, Perkins NJ, Platt RW. Quantification of collider-stratification bias and the birthweight paradox. Paediatric and Perinatal Epidemiology. 2009;23:394-402.

The pitfall of considering design effect during the design phase of a study but ignoring it during the analysis

Nowadays, it is a common practice to employ complex sampling strategies—sampling strategies that involve stratification, clustering and weighting—in public health survey researches. This is so because the use of simple random sampling (SRS) in population-based surveys imposes both technical and financial difficulties. Besides, objectives of a study such as making sub-population inferences may necessitate using such sampling schemes.

In many public health surveys implementing the foregoing sampling strategy, it is common to see the sample size needed if SRS were used multiplied by an anticipated design effect (i.e., the measure of loss of precision due to complex sampling). That is in fact a necessary step in studies planning complex sampling procedures.

But sadly enough, in many such studies the sampling scheme is completely forgotten during the analysis phase of the study and data are analyzed as if they were generated through SRS technique. In effect, analysis outputs of interest such population descriptive statistics and model parameters are biased.

Briefly, here is what happens due to failure to take account of the sampling features during the analysis.

1. Sampling weights help to attenuate bias that results from unequal probability of selection of sampling units. Thus, ignoring weighting during the analysis results in underestimation of population descriptive statistics such as means and proportions.
2. Clustering generally decrease precision (i.e., increases the standard error of estimates). Hence, failing to take account of clustering during the analysis underestimates the standard error (because such analyses are based on assumption of SRS). In effect, the α-error is inflated.
3. Stratification increases the efficiency of the sample. That means, it reduces the standard error of estimates. When it is ignored during analysis, the standard error is increased. In effect, β-error is inflated.

The joint effect of weighting, clustering and stratification is a tug-of-war with weighting and clustering on one side pulling the standard error up and stratification on the other side pulling the standard error down. The net effect depends on which side succeeds in dragging the other. Often, the pulling effect of clustering and weighting outweighs and standard errors are increased as a net effect. Thus, all sampling features should be explicitly taken account of in the analysis of data obtained based on the complex sampling scheme. Otherwise, the bias becomes paramount.

The pitfall of failing to take account of design features in studies that have considered design effect during the sample size calculation is this. The sample size required to attain a given precision using a complex sample is much higher than the corresponding SRS sample size. Hence, when the sampling features are ignored during the analysis, an SRS of large sample size is wrongly assumed. Thus, the standard errors of the estimates are highly underestimated (though there is also a possibility of overestimation based on the net effect of stratification, clustering and weighting). As a result, statistical tests are likely to be significant while they are not (i.e., tests of significance are biased away from the null; the possibility of bias towards the null cannot be precluded, however.)

To conclude, I would say instead of considering design effect during the sampling design and then ignore all design features during the analysis, it may be better to ignore the design effect from the outset. Or else, analysis techniques that properly take account of the sampling scheme should be applied.

Reference

For a detailed treatment of the complex sampling techniques and associated analytic issues readers are referred to:

• Heeringa SG, West BT, Berglund PA. Applied Survey Data Analysis. Boca Raton, FL: Chapman & Hall/CRC; 2010.