Creating an Incomplete Block Design in R

From Survey Analysis
Jump to: navigation, search

Contents

Step 1: Install the AlgDesign package

The AlgDesign package needs to be installed in R.

Step 2: Load this function into R

Copy and paste the following function into R.

incomplete.block.design <- function(number.alternatives, number.blocks, alternatives.per.block, n.repeats = 1000){
  library(AlgDesign)
  best.result = NULL
  best.D = -Inf
  for (i in 1:n.repeats){
    alg.results <- optBlock(~.,withinData=factor(1:number.alternatives),blocksizes=rep(alternatives.per.block,number.blocks), nRepeats=5000) #BIB
    if (alg.results$D > best.D){
      best.results = alg.results
      best.D = alg.results$D
    }
  }
  design <- matrix(NA,number.blocks,alternatives.per.block, dimnames= list(block = 1:number.blocks, Alternative = 1:alternatives.per.block))
  binary.design <- matrix(0,number.blocks,number.alternatives, dimnames= list(block = 1:number.blocks, alternative = 1:number.alternatives))
  counter <- 0
  for (block in alg.results$Blocks){
    counter <- counter + 1
    blck <- unlist(block)
    design[counter,] <- blck
    for (a in blck)
      binary.design[counter,a] <- 1
  }
  n.appearances.per.alternative <- table(as.numeric(design))
  combinations.of.alternatives <- crossprod(table(c(rep(1:number.blocks, rep(alternatives.per.block,number.blocks))),alg.results$design[,1]))
  list(binary.design = t(binary.design), design = t(design), frequencies = n.appearances.per.alternative, pairwise.frequencies=combinations.of.alternatives, binary.correlations = round(cor(binary.design),2))}

(This code has benefited from some improvements suggested by Joel Cadwell).

Step 3: Run the function

Type or paste the following into R, modifying the number.alternatives, number.blocks and alternatives.per.block to suit your needs. The n.repeats parameter determines how many attempts are made at finding an optimum design.

incomplete.block.design(number.alternatives = 10, number.blocks = 11, alternatives.per.block = 5, n.repeats = 100)

Step 4: Interpreting the design outputs

The first output is a table showing the design as binary variables (refer to Max-Diff Experimental Design for an example of this table's interpretation).

$binary.design
           block
alternative 1 2 3 4 5 6 7 8 9 10 11
         1  1 1 0 0 1 1 0 0 0  1  0
         2  1 1 1 0 1 0 1 0 0  0  1
         3  0 1 1 0 0 1 1 1 0  0  0
         4  1 0 0 0 0 0 1 1 1  1  1
         5  0 1 1 1 0 0 0 0 1  1  1
         6  0 0 0 1 1 1 1 0 0  1  1
         7  0 1 0 1 1 0 0 1 1  0  0
         8  1 0 1 1 0 0 0 1 0  1  0
         9  1 0 0 1 0 1 1 0 1  0  0
         10 0 0 1 0 1 1 0 1 1  0  1

The next output contains the same information as the binary.design but in a different form that is sometimes more straightforward for use when preparing the Experimental Protocol. Looking at the last column, for example, it shows that in the 11th block respondents are to be shown alternatives 2, 4, 5, 6 and 10.

           block
Alternative 1 2  3 4  5  6 7  8  9 10 11
          1 1 1  2 5  1  1 2  3  4  1  2
          2 2 2  3 6  2  3 3  4  5  4  4
          3 4 3  5 7  6  6 4  7  7  5  5
          4 8 5  8 8  7  9 6  8  9  6  6
          5 9 7 10 9 10 10 9 10 10  8 10

The frequencies indicate the number of times each alternative appears in the experiment.


$frequencies

 1  2  3  4  5  6  7  8  9 10 
 5  6  5  6  6  6  5  5  5  6 

The pairwise frequencies show the number of times that each alternative appears with each other alternative. Note that the Main Diagonal is a the same as the frequencies.


$pairwise.frequencies
    
     1 2 3 4 5 6 7 8 9 10
  1  5 3 2 2 2 3 2 2 2  2
  2  3 6 3 3 3 3 2 2 2  3
  3  2 3 5 2 2 2 2 2 2  3
  4  2 3 2 6 3 3 2 3 3  3
  5  2 3 2 3 6 3 3 3 2  3
  6  3 3 2 3 3 6 2 2 3  3
  7  2 2 2 2 3 2 5 2 2  3
  8  2 2 2 3 3 2 2 5 2  2
  9  2 2 2 3 2 3 2 2 5  2
  10 2 3 3 3 3 3 3 2 2  6

The final output is a Correlation Matrix showing the correlations between the columns in the first of the outputs. Note that with incomplete block designs it is not possible for the correlations of the Off-Diagonal entries to be 0.


$binary.correlations
       1     2     3     4     5     6     7     8     9    10
1   1.00  0.10 -0.10 -0.27 -0.27  0.10 -0.10 -0.10 -0.10 -0.27
2   0.10  1.00  0.10 -0.10 -0.10 -0.10 -0.27 -0.27 -0.27 -0.10
3  -0.10  0.10  1.00 -0.27 -0.27 -0.27 -0.10 -0.10 -0.10  0.10
4  -0.27 -0.10 -0.27  1.00 -0.10 -0.10 -0.27  0.10  0.10 -0.10
5  -0.27 -0.10 -0.27 -0.10  1.00 -0.10  0.10  0.10 -0.27 -0.10
6   0.10 -0.10 -0.27 -0.10 -0.10  1.00 -0.27 -0.27  0.10 -0.10
7  -0.10 -0.27 -0.10 -0.27  0.10 -0.27  1.00 -0.10 -0.10  0.10
8  -0.10 -0.27 -0.10  0.10  0.10 -0.27 -0.10  1.00 -0.10 -0.27
9  -0.10 -0.27 -0.10  0.10 -0.27  0.10 -0.10 -0.10  1.00 -0.27
10 -0.27 -0.10  0.10 -0.10 -0.10 -0.10  0.10 -0.27 -0.27  1.00

Step 5: Checking the design

  1. Check that there are no large correlations in the off-diagonal cells of the binary correlations. All else being equal, the more uniform the correlations the better.
  2. Conduct a Soft Send and estimate the required models (or, if this is not practical, invent some fake data and check), paying particular attention to the standard errors of any estimates from an aggregate model). This step simultaneously checks that the experimental design is sound, that experimental design is suitable for the statistical model and that the experiment has been correctly implemented.

Step 6: Randomizing

There is always the potential that non-obvious biases exist within a specific design. For example:

  • It could be that certain combinations of 3 or more alternatives have some unforeseen effect and that these combinations are particularly prevalent or or absent within a design.
  • The ordering with which the alternatives appear may cause a bias, with people being more likely to prefer alternatives at a certain spot on the page.

A way of reducing such biases is to randomize between respondents. That is, use the same design for each respondent, but randomize the order of the rows of the design when assigning alternatives (i.e., so the design for one respondent's first alternative is the design for another respondents fifth alternative). Similar, randomizing the order of the sets may also be beneficial.

See also

Personal tools
Namespaces

Variants
Actions
Navigation
Categories
Toolbox