Guttman's lambda2 for estimating reliability in stata?

[PvanKessel]

Hi all, I would like to calculate Guttman's lambda2, as this is a more accurate estimate of the reliability of a scale (see attached article). However, this function appears not to be available in Stata 14.1. Is there any work-around or package/ado that anyone knows of? If so, it would be more than welcome. Kind regards, Paul van Kessel, The Netherlands

[Joseph Coveney]

You could try something like that below (start at the "Begin here" comment). Although the program used below is off-the-cuff and so I don't warrant anything, I've attached it as an ado-file if you're interested in pursuing things further on your own. To help you there, the program uses the same nomenclature as in the first two equations of John C. Callender and H. G. Osburn, An Empirical Comparison of Coefficient Alpha, Guttman's Lambda - 2, and MSPLIT Maximized Split-Half Reliability Estimates. Journal of Educational Measurement 16(2):89–99, 1979 (All I am able to see of the article is a .png image of the first page, but if you're affiliated with a university, then you should be able to see the entire thing.). The program uses listwise deletion of incomplete observations.

.ÿversionÿ14.1

.ÿ
.ÿclearÿ*

.ÿsetÿmoreÿoff

.ÿtempnameÿCorr

.ÿmatrixÿdefineÿ`Corr'ÿ=ÿJ(6,ÿ6,ÿ0.5)ÿ+ÿI(6)ÿ*ÿ0.5

.ÿquietlyÿdrawnormÿy1ÿy2ÿy3ÿy4ÿy5ÿy6,ÿdoubleÿcorr(`Corr')ÿn(200)ÿseed(`=date("2016-01-06",ÿ"YMD")')

.ÿ
.ÿ*
.ÿ*ÿBeginÿhere
.ÿ*
.ÿprogramÿdefineÿglambda2,ÿrclass
ÿÿ1.ÿÿÿÿÿÿÿÿÿversionÿ14.1
ÿÿ2.ÿÿÿÿÿÿÿÿÿsyntaxÿvarlist(numeric)ÿ[if]ÿ[in]
ÿÿ3.ÿ
.ÿÿÿÿÿÿÿÿÿmarksampleÿtouse
ÿÿ4.ÿÿÿÿÿÿÿÿÿ
.ÿÿÿÿÿÿÿÿÿ//ÿGuttmanÿLambdaÿ1
.ÿÿÿÿÿÿÿÿÿtempvarÿscale_score
ÿÿ5.ÿÿÿÿÿÿÿÿÿquietlyÿegenÿdoubleÿ`scale_score'ÿ=ÿrowtotal(`varlist')ÿifÿ`touse'
ÿÿ6.ÿÿÿÿÿÿÿÿÿquietlyÿsummarizeÿ`scale_score'ÿifÿ`touse',ÿdetail
ÿÿ7.ÿÿÿÿÿÿÿÿÿtempnameÿsigma2x
ÿÿ8.ÿÿÿÿÿÿÿÿÿscalarÿdefineÿ`sigma2x'ÿ=ÿr(Var)
ÿÿ9.ÿÿÿÿÿÿÿÿÿ
.ÿÿÿÿÿÿÿÿÿquietlyÿcorrelateÿ`varlist'ÿifÿ`touse',ÿcovariance
ÿ10.ÿÿÿÿÿÿÿÿÿtempnameÿC
ÿ11.ÿÿÿÿÿÿÿÿÿmatrixÿdefineÿ`C'ÿ=ÿr(C)
ÿ12.ÿÿÿÿÿÿÿÿÿtempnameÿsum_sigma2j
ÿ13.ÿÿÿÿÿÿÿÿÿscalarÿdefineÿ`sum_sigma2j'ÿ=ÿtrace(`C')
ÿ14.ÿ
.ÿÿÿÿÿÿÿÿÿtempnameÿlambda_1
ÿ15.ÿÿÿÿÿÿÿÿÿscalarÿdefineÿ`lambda_1'ÿ=ÿ1ÿ-ÿ`sum_sigma2j'ÿ/ÿ`sigma2x'
ÿ16.ÿ
.ÿÿÿÿÿÿÿÿÿ//ÿGuttmanÿLambdaÿ2
.ÿÿÿÿÿÿÿÿÿtempnameÿgamma_2
ÿ17.ÿÿÿÿÿÿÿÿÿscalarÿdefineÿ`gamma_2'ÿ=ÿ0
ÿ18.ÿÿÿÿÿÿÿÿÿlocalÿnÿ=ÿcolsof(`C')
ÿ19.ÿÿÿÿÿÿÿÿÿforvaluesÿiÿ=ÿ1/`=`n'-1'ÿ{
ÿ20.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿforvaluesÿjÿ=ÿ`=`i'+1'/`n'ÿ{
ÿ21.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿscalarÿdefineÿ`gamma_2'ÿ=ÿ`gamma_2'ÿ+ÿ`C'[`i',ÿ`j']
ÿ22.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ}
ÿ23.ÿÿÿÿÿÿÿÿÿ}
ÿ24.ÿ
.ÿÿÿÿÿÿÿÿÿ
.ÿÿÿÿÿÿÿÿÿreturnÿscalarÿlambda_2ÿ=ÿ`lambda_1'ÿ+ÿsqrt(`n'ÿ/ÿ(`n'-1)ÿ*ÿ`gamma_2')ÿ/ÿ`sigma2x'
ÿ25.ÿ
.ÿÿÿÿÿÿÿÿÿdisplayÿinÿsmclÿasÿtextÿ"GuttmanÿLambdaÿ2ÿ=ÿ"ÿasÿresultÿ%06.4fÿreturn(lambda_2)
ÿ26.ÿend

.ÿ
.ÿglambda2ÿy?
GuttmanÿLambdaÿ2ÿ=ÿ0.8650

.ÿalphaÿy?

Testÿscaleÿ=ÿmean(unstandardizedÿitems)

Averageÿinteritemÿcovariance:ÿÿÿÿÿ.4719189
Numberÿofÿitemsÿinÿtheÿscale:ÿÿÿÿÿÿÿÿÿÿÿÿ6
Scaleÿreliabilityÿcoefficient:ÿÿÿÿÿÿ0.8608

.ÿ
.ÿexit

endÿofÿdo-file


.

If you haven't done so already, you might want to take a look at Stata's sem (or gsem and irt as approproate).

[daniel klein]

I think there is a small bug in Joseph's code, as it sums up the covariances, not their squares and also does so only for half of them. To correct this, first line 21 above needs to read

scalar define `gamma_2' = `gamma_2' + `C'[`i', `j']^2

Since covariances are symmetric, gamma2 should then be multiplied by a factor of two.

Here is my try, drawing on the original paper by Guttman (1945). As with Joseph's code only complete observations are used and no attention is payed to the sign of the variables when computing the score. If I find the time I will address these issues and also add the remaining three lambda coefficients.

*! version 1.0.0 06jan2016 daniel klein
pr guttmanl // , rclass
    vers 11.2
    
    syntax varlist(min = 2 num) [if] [in]
    
    marksample touse
    qui cou if `touse'
    if (r(N) < 2) {
        err 2001
    }
    
    m : guttmanl()
    
    di as txt _n "Guttman lower bound reliability coefficients" _n
    di as txt "Guttman lambda1: " as res %06.4f r(lambda1)
    di as txt "Guttman lambda2: " as res %06.4f r(lambda2)
    di as txt "Guttman lambda3: " as res %06.4f r(lambda3)
end

vers 11.2

m : 

void guttmanl()
{
    real matrix X, Var, Cov
    real colvector sscore
    real scalar n, sum_sj2, st2, l1, l2, l3
    
    X = st_data(., st_local("varlist"), st_local("touse"))
    n = cols(X)
    
    Var = Cov = variance(X)
    sum_sj2 = colsum(diagonal(Var))
    _diag(Cov, 0)
    
    sscore = rowsum(X)
    st2 = variance(sscore)
    
    l1 = 1 - (sum_sj2/st2)
    l2 = l1 + sqrt(n/(n - 1)*sum(Cov:^2))/st2
    l3 = n/(n - 1)*l1
    
    st_rclear()
    st_numscalar("r(lambda1)", l1)
    st_numscalar("r(lambda2)", l2)
    st_numscalar("r(lambda3)", l3)
}

end

Best
Daniel


Guttman, L. 1945. A BASIS FOR ANALYZING TEST-RETEST RELIABILITY. Psychometrika, 10(4), 255-282.

[Joseph Coveney]

Thanks, Daniel, for the catch.

I've corrected the program and made it more efficient, getting rid of the for-loops altogether and it no longer relies on egen, correlation and other commands for calculating intermediate values and for error-trapping.

It's also now split into two ado-files, the second of which is able to use a covariance matrix (or correlation matrix and standard deviation vector) directly. Because official alpha doesn't do that, I thought that it might be worth making them available and attached them.

glambda2 is for use with datasets; glambda2mat is for use with a correlation or covariance matrix. alpha is an option to provide Guttman's lambda 3, probably most useful with the second, given the availability of the official Stata command alpha otherwise. Like before, these have listwise deletion and do not do sign reversal. Use of the two is illustrated below to give an idea of their syntax for anyone interested.

.ÿversionÿ14.1

.ÿ
.ÿclearÿ*

.ÿsetÿmoreÿoff

.ÿ
.ÿ*
.ÿ*ÿUseÿwithÿaÿdataset
.ÿ*
.ÿtempnameÿCorr

.ÿmatrixÿdefineÿ`Corr'ÿ=ÿJ(6,ÿ6,ÿ0.5)ÿ+ÿI(6)ÿ*ÿ0.5

.ÿquietlyÿdrawnormÿy1ÿy2ÿy3ÿy4ÿy5ÿy6,ÿdoubleÿcorr(`Corr')ÿn(200)ÿseed(`=date("2016-01-06",ÿ"YMD")')

.ÿ
.ÿglambda2ÿy?,ÿalpha
(obs=200)
Guttman'sÿLambdaÿ2ÿ=ÿ0.8614
ÿÿCronbach'sÿalphaÿ=ÿ0.8608

.ÿalphaÿy?

Testÿscaleÿ=ÿmean(unstandardizedÿitems)

Averageÿinteritemÿcovariance:ÿÿÿÿÿ.4719189
Numberÿofÿitemsÿinÿtheÿscale:ÿÿÿÿÿÿÿÿÿÿÿÿ6
Scaleÿreliabilityÿcoefficient:ÿÿÿÿÿÿ0.8608

.ÿ
.ÿ*
.ÿ*ÿUseÿwithÿdirectÿmatrixÿinput:ÿcorrlationÿmatrixÿ+ÿvectorÿofÿSDs
.ÿ*
.ÿglambda2matÿ(1ÿ0.5ÿ0.5ÿ\ÿ0.5ÿ1ÿ0.5ÿ\ÿ0.5ÿ0.5ÿ1),ÿsd(2ÿ3ÿ4)ÿalpha
Guttman'sÿLambdaÿ2ÿ=ÿ0.7187
ÿÿCronbach'sÿalphaÿ=ÿ0.7091

.ÿ
.ÿmatrixÿinputÿ`Corr'ÿ=ÿ(1ÿ0.5ÿ0.5ÿ\ÿ0.5ÿ1ÿ0.5ÿ\ÿ0.5ÿ0.5ÿ1)

.ÿmatrixÿinputÿSDÿ=ÿ(2ÿ3ÿ4)

.ÿglambda2matÿ`Corr',ÿsd(SD)ÿalpha
Guttman'sÿLambdaÿ2ÿ=ÿ0.7187
ÿÿCronbach'sÿalphaÿ=ÿ0.7091

.ÿ
.ÿ*
.ÿ*ÿUseÿwithÿdirectÿmatrixÿinput:ÿcovarianceÿmatrix
.ÿ*
.ÿmatrixÿdefineÿCovarianceÿ=ÿ(4,ÿ3,ÿ4ÿ\ÿ3,ÿ9,ÿ6ÿ\ÿ4,ÿ6,ÿ16)

.ÿglambda2matÿCovariance,ÿalpha
Guttman'sÿLambdaÿ2ÿ=ÿ0.7187
ÿÿCronbach'sÿalphaÿ=ÿ0.7091

.ÿ
.ÿ*
.ÿ*ÿUseÿwithÿcategoricalÿdata:ÿbinaryÿresponses
.ÿ*
.ÿquietlyÿwebuseÿfamilyvalues,ÿclear

.ÿquietlyÿtetrachoricÿRS074ÿRS075ÿRS076

.ÿglambda2matÿr(Rho),ÿalpha
Guttman'sÿLambdaÿ2ÿ=ÿ0.6339
ÿÿCronbach'sÿalphaÿ=ÿ0.5852

.ÿ
.ÿ//ÿTrappingÿnotÿpositive-semidefiniteÿmatrixÿ(exampleÿfromÿ-tetrachoric-ÿhelpÿfile)
.ÿquietlyÿtetrachoricÿRS056-RS063ÿinÿ1/20

.ÿcaptureÿnoisilyÿglambda2matÿr(Rho)
Matrixÿnotÿatÿleastÿpositiveÿsemidefinite

.ÿ
.ÿquietlyÿtetrachoricÿRS056-RS063ÿinÿ1/20,ÿposdef

.ÿglambda2matÿr(Rho)
Guttman'sÿLambdaÿ2ÿ=ÿ0.5119

.ÿ
.ÿ*
.ÿ*ÿUseÿwithÿcategoricalÿdata:ÿorderedÿcategoricalÿresponses
.ÿ*
.ÿquietlyÿdrawnormÿy1ÿy2ÿy3,ÿdoubleÿcorr(1ÿ0.7ÿ0.3ÿ\ÿ0.7ÿ1ÿ0.7ÿ\ÿ0.3ÿ0.7ÿ1)ÿn(200)ÿclear

.ÿforvaluesÿiÿ=ÿ1/3ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿgenerateÿbyteÿo`i'ÿ=ÿ1
ÿÿ3.ÿÿÿÿÿÿÿÿÿforvaluesÿcutÿ=ÿ0.2(0.2)0.8ÿ{
ÿÿ4.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿquietlyÿreplaceÿo`i'ÿ=ÿo`i'ÿ+ÿ1ÿifÿnormal(y`i')ÿ>ÿ`cut'
ÿÿ5.ÿÿÿÿÿÿÿÿÿ}
ÿÿ6.ÿ}

.ÿ
.ÿ//ÿConstructingÿpolychoricÿcorrelationÿmatrixÿusingÿofficialÿStataÿcommands
.ÿmatrixÿinputÿRhoÿ=ÿ(1ÿ.ÿ.ÿ\ÿ.ÿ1ÿ.ÿ\ÿ.ÿ.ÿ1)

.ÿforvaluesÿiÿ=ÿ1/2ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿforvaluesÿjÿ=`=`i'+1'/3ÿ{
ÿÿ3.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿquietlyÿgsemÿ(o`i'@1ÿo`j'@1ÿ<-ÿF,ÿoprobit),ÿmeans(F@0)ÿnocnsreportÿnodvheaderÿnolog
ÿÿ4.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmatrixÿdefineÿRho[`i',ÿ`j']ÿ=ÿsqrt(_b[var(F):_cons])ÿ/ÿ2
ÿÿ5.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmatrixÿdefineÿRho[`j',ÿ`i']ÿ=ÿRho[`i',ÿ`j']
ÿÿ6.ÿÿÿÿÿÿÿÿÿ}
ÿÿ7.ÿ}

.ÿ
.ÿglambda2matÿRho,ÿalpha
Guttman'sÿLambdaÿ2ÿ=ÿ0.8606
ÿÿCronbach'sÿalphaÿ=ÿ0.8419

.ÿ
.ÿexit

endÿofÿdo-file


.

[PvanKessel]

Thanks all for your efforts!

[Joseph Coveney]

A final correction, this time on the computation of the polychroic correlation coefficients.

.ÿversionÿ14.1

.ÿ
.ÿclearÿ*

.ÿsetÿmoreÿoff

.ÿ
.ÿ*
.ÿ*ÿUseÿwithÿcategoricalÿdata:ÿorderedÿcategoricalÿresponses
.ÿ*
.ÿtempnameÿCorr

.ÿmatrixÿdefineÿ`Corr'ÿ=ÿJ(6,ÿ6,ÿ0.5)ÿ+ÿI(6)ÿ*ÿ0.5

.ÿquietlyÿdrawnormÿy1ÿy2ÿy3ÿy4ÿy5ÿy6,ÿdoubleÿcorr(`Corr')ÿn(200)ÿseed(`=date("2016-01-12",ÿ"YMD")')

.ÿ
.ÿforvaluesÿiÿ=ÿ1/6ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿgenerateÿbyteÿo`i'ÿ=ÿ1
ÿÿ3.ÿÿÿÿÿÿÿÿÿforvaluesÿcutÿ=ÿ0.2(0.2)0.8ÿ{
ÿÿ4.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿquietlyÿreplaceÿo`i'ÿ=ÿo`i'ÿ+ÿ1ÿifÿnormal(y`i')ÿ>ÿ`cut'
ÿÿ5.ÿÿÿÿÿÿÿÿÿ}
ÿÿ6.ÿ}

.ÿ
.ÿ//ÿConstructingÿpolychoricÿcorrelationÿmatrixÿusingÿofficialÿStataÿcommands
.ÿmatrixÿdefineÿRhoÿ=ÿJ(6,ÿ6,ÿ1)

.ÿforvaluesÿiÿ=ÿ1/5ÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿforvaluesÿjÿ=`=`i'+1'/6ÿ{
ÿÿ3.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿquietlyÿgsemÿ(o`i'@1ÿo`j'@1ÿ<-ÿF,ÿoprobit),ÿmeans(F@0)ÿnocnsreportÿnodvheaderÿnolog
ÿÿ4.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmatrixÿdefineÿRho[`i',ÿ`j']ÿ=ÿ_b[var(F):_cons]ÿ/ÿ(1ÿ+ÿ_b[var(F):_cons])
ÿÿ5.ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿmatrixÿdefineÿRho[`j',ÿ`i']ÿ=ÿRho[`i',ÿ`j']
ÿÿ6.ÿÿÿÿÿÿÿÿÿ}
ÿÿ7.ÿ}

.ÿ
.ÿglambda2matÿRho,ÿalpha
Guttman'sÿLambdaÿ2ÿ=ÿ0.8746
ÿÿCronbach'sÿalphaÿ=ÿ0.8737

.ÿ
.ÿalphaÿy?

Testÿscaleÿ=ÿmean(unstandardizedÿitems)

Averageÿinteritemÿcovariance:ÿÿÿÿÿ.5451882
Numberÿofÿitemsÿinÿtheÿscale:ÿÿÿÿÿÿÿÿÿÿÿÿ6
Scaleÿreliabilityÿcoefficient:ÿÿÿÿÿÿ0.8731

.ÿ
.ÿexit

endÿofÿdo-file


.

It's surprising how close Cronbach's alpha is between that calculated from the underlying (latent) variables and that recovered in the polychoric correlations after discretization (albeit five categories).

[daniel klein]

Thanks to Kit Baum, a revised version of guttmanl is now available from the SSC archives. In Stata type

ssc install guttmanl

to install this.

The program now

• calculates all six reliability coefficients proposed by Guttman (1945)
• instead of using complete cases only (default), optionally bases calculations on pairwise observations (not recommended) or obtains the covariance matrix from an EM algorithm
• allows specifying revered signs for variables or determines their signs empirically
• accepts a covariance (or correlation) matrix as input

I hope some will find this useful.

Best
Daniel


Guttman, L. 1945. A BASIS FOR ANALYZING TEST-RETEST RELIABILITY. Psychometrika, 10(4), 255-282.

[PvanKessel]

Thanks to Kit Baum, a revised version of guttmanl is now available from the SSC archives. In Stata type

ssc install guttmanl

to install this.

The program now

• calculates all six reliability coefficients proposed by Guttman (1945)
• instead of using complete cases only (default), optionally bases calculations on pairwise observations (not recommended) or obtains the covariance matrix from an EM algorithm
• allows specifying revered signs for variables or determines their signs empirically
• accepts a covariance (or correlation) matrix as input

I hope some will find this useful.

Best
Daniel


Guttman, L. 1945. A BASIS FOR ANALYZING TEST-RETEST RELIABILITY. Psychometrika, 10(4), 255-282.

Looks great! Thanks.

[daniel klein]

Thanks once again to Kit Baum, an updated version of guttmanl is now available from the SSC.

The update concerns the estimation of split-half reliability (lambda 4). There are two new alternatives to the default method of considering every possible split, which is slow for a large number of items (think more than 15) and might hit memory limits when used with even larger number of items (think more than 20). guttmanl now allows one user-defined split of items (e.g. odd-even split) and, probably more relevant, estimates quantiles of a random sereis of locally optimal split-half coefficients. This latter approach is suggested by Hunt and Benler (2015) who argue that it is feasible with a large number of items and provides less upwardly biased estimates in small samples. It is implemented via the new quantiles option.

Best
Daniel


Hunt, T.D., Bentler, P.M. (2015). Quantile Lower Bounds to Population Reliability Based on Locally Optimal Splits. Psychometrika, 80(1), 182-195.