HLM (Hierarchical Linear and Nonlinear Modeling) is used for the statistical modeling of two and three-level data structures.

Using SAS PROC MIXED to fit multilevel, hierarchical and individual growth models.

A tutorial that shows how to use SAS to fit the two most common multilevel models.

Fisher scoring algorithm is unable to produce acceptable estimates

The message

The Fisher scoring algorithm is unable to produce acceptable estimates of the variance-covariance components. It is possible that the model you have fitted is too complex for the data at hand

is only applicable to the non-unrestricted models available with HMLM and HMLM2. The unrestricted iterations use EM as an estimation method, and therefore cannot iterate to impossible (i.e. negative) variances. The other types of iterations use a Fisher scoring method, and can arrive at a solution outside the parameter space. The programs will attempt to control this problem, but only up to a point, after which it will stop and produce the message given above. The cause of this is usually an element of the random effect variance-covariance matrix (see the D matrix) being very close to 0. In the HMLM/HMLM2 case, though, it could also be one of the other "extra" parameters.


Group X has inadequate data

Description of the problem

This message appears after the Make SSM button is clicked. It is listed in the DOS box that appears on the screen during the SSM file creation process when listwise deletion of records with missing data is requested at the SSM file creation stage of the analysis. The groups listed in the DOS box have insufficient data, and are not included in the SSM file. This is usually a function of the inclusion of a large number of variables in the data file with a large number of missing data values.

Solution

Check the structure of the particular level-2 units listed and the distributions of the level-1 variables. If there are no valid data points for a level-2 or level-3 unit when doing simple frequencies with a listwise deletion option in a statistical package, that unit will be one of those deleted when the SSM file is created. It can be remedied by removing the variable(s) with the most missing data, or else imputing the missing values for these variables. A list of such units is also printed to the *.sts file, which may be accessed using the Check Stats button in the Make SSM dialog box.


Group X not in level-y file

Description

A warning message is written to the DOS box when the values of the ID variables used in the data files for the different levels of the hierarchy do not correspond. It identifies the value for which a match could not be found in the ID variable field in the other data file(s). The absence of a unit will also be apparent from the descriptive statistics in either the DOS box or the default *.sts file, where the number of units used will be less than anticipated. A list of such units is also printed to the *.sts file, which may be accessed using the Check Stats button in the Make SSM/MDM dialog box.


Solution

Frequency tables of the ID codes used in the different data files can be used to verify that all ID codes are present in all files. Also check that the ID variables have exactly the same format across all data files.


HLM is unable to estimate covariance components for the model specified

Description

HLM is unable to estimate covariance components for the model specified. It is likely that either

  1. one or more of the variance components is very close to zero and the reliability of the associated random effect is also close to zero, or

  2. there is a collinearity or multicollinearity among the random effects. In this case, the estimated correlations among the random effects would beclose to 1.0, or

  3. one or more of the OLS level-1 regressions produced extreme values.

Solution

  • To check option (1), the tau-matrix printed in the output file must be examined. Small values on the diagonal of this matrix indicate the variable causing this problem.

  • The information needed to check option (2) follows directly after the tau-matrix, where the tau-matrix is given in the form of a correlation matrix.

  • Review the OLS estimates for all groups to find problems associated with option

  • If this is the source of the problem, use the option to manually reset the tau(0) matrix on the Basic Specifications menu.

Deciding which level-1 effect to keep random and which to change to non-randomly varying should be based on theory and research purposes.


Laplace iterations stops due to bad tau

Description

In some cases when Laplace is used as method of estimation for non-linear models, the iterative procedure will terminate prematurely with a message indicating that it is unable to continue due to a "bad tau".

Generally speaking, a "bad tau" is a covariance matrix with very small or even negative variances (diagonal elements). The continuation of the iterative procedure is dependent on the inversion of the tau matrix and as a matrix with elements as described above does not have an unique inverse, this causes problems. In the case of standard Expected Maximization (EM) runs, the program will endlessly try to fix such a tau by adding small values or constraining off-diagonal elements to zero. The result is quite often that the procedure fails to reach convergence, regardless of the maximum number of iterations allowed. In the case of Laplace iterations, however, this does not happen. The moment a tau matrix which can not be converted is encountered, the program will stop with a message referring to a "bad tau".
 

The cause of this message can usually be found by inspecting the tau matrix given in the output file. Small or negative diagonal elements in the tau matrix, or high correlations in the tau (as correlations) matrix are likely causes.
 

Solution

Try centering the predictors involved or removing the random effects associated with the diagonal elements of tau causing the problem.


OLS level-1 coefficients were computed for only x of y units that had sufficient data for estimation

This message appears in the output file in the section where the OLS coefficients and starting values are given. It may also appear at the tables containing reliabilities, and at the tables with the final fixed and random effects. In all cases it is relevant only for the section of the output where the message is printed.

Note that when it appears at the final tables, it affects the chi-square values printed in these tables and generalization on the basis of these chi-squares are inadvisable, especially if a sizable proportion of the units is excluded during the chi-square calculations.

The usual cause of this message is a lack of variability within higher level units. For example, consider level-2 model with schools as the level-2 units and students as the level-1 units. If, for a given school, all students are of the same gender this may cause the problem. The following error message may also subsequently appear.

There is a problem in the fixed portion of the model. A near singularity is likely. Possible sources are a collinearity or multicollinearity among the predictors. We suggest that you examine a correlation matrix among the fixed effect predictors

This error message indicates that there are some of the fixed effects that have essentially the same relationship with the outcome variable.


Only X out of Y units used in analysis

Description

This message may appear at various places in the output file, and only applies to the specific section of the output where it appears. It is important to realize no cases are really "dropped". Certain statistics printed by HLM require the least squares estimator to exist for an unit to be included. These include the univariate chi-square tests, reliabilities and least-squares estimates. For these statistics and these only, some cases are omitted. However, the main results (fixed effects, that is- the gammas), variance and covariance components (the tau's) and empirical Bayes estimates, as well as tests for all coefficients and all standard errors- are based on all the data.

In the example below, an example is given for a level-2 unit with 5 level-1 units nested within the level-2 unit. The first column represents the intercept term, which is by default included in any HLM model. The second column represents the scores of the 5 respondents from this level-2 unit. As the scores of all 5 respondents are very similar, the second column is almost a multiple of the first.


Intercept Score

1 20

1 20

1 20

1 20

1 21


Groups that passed the various checks on X'X (X = level-1 data), invertibility, positive determinant, condition number < 1E6, will have the OLS coefficients printed out. Groups that fail either of the first two checks will not appear, and will be counted as insufficient data. Groups that fail the condition check will be counted, and produce the message noted above. The level-1 coefficients for all "acceptable" units can be requested from the Output Settings dialog box (accessible from the Other Settings menu) by setting that field to the number of level-2 groups (or some large number). The results for "acceptable" groups will be printed. Note that this option is only available for HLM2.

This can also be caused by a lack of sufficient data for certain groups. For example, let’s say we have a model with 3 random effects. Groups with less than 3 level-1’s per level-2 (using HLM2 as an example) will cause this error, and probably at least some of the groups with three records will as well.

Solution

Check the variability of level-1 predictor values within each level-2 unit. Level-2 units that have time-varying covariates that are actually constant are the most likely cause for the display of the above warning.


Robust standard errors cannot be computed for this model

For some models, the message

Robust standard errors cannot be computed for this model may appear in the sections of the output file pertaining to fixed effects.

The robust standard errors should be trusted only when the number of higher-level units is moderately large relative to the number of explanatory variables at a higher level. At some point, the number of higher-level units can become so small that these standard errors are not computable as the information matrix is uninvertible or not positive-definite. In such cases, a message that robust standard errors could not be computed for a model may be printed to the HLM output file.


SSM file not of recognized type

Description

The appearance of this message is usually an indication that there are problems with the SSM file created for use in analysis. If, for example, missing data were incorrectly specified, a SSM file may be created. It may, however, be an empty file that contains no information that can be used by HLM in model building.

Solution

The best way to solve this problem is to remake the SSM file and to pay close attention to the message printed to the DOS box during SSM file creation. Use the Check Stats button in the Make SSM dialog box to access the descriptive statistics for the data and verify that these statistics correspond to the results of prior exploratory data analysis. Any error message concerning missing data has to be attended to before model building is attempted.


The descriptive statistics produced by HLM are incorrect

The descriptive statistics for data read by HLM are both printed to screen and to the default statistics file:

hlm2ssm.sts for a level-2 model (using HLM2)

hlm3ssm.sts for a level-3 model (using HLM3)

hmlmmdm.sts for a 2-level multivariate model (using HMLM)

hmlm2mdm.sts for a 3-level multivariate model (using HMLM2)

When these statistics do not correspond at all with those obtained during exploratory data analysis, it is caused by one of the following two problems:

  • Whether ASCII or statistical package data are used, this message is caused by the incorrect specification of missing data. Go back to the original data file in the statistical package used to construct it, and check the missing data specification for each predictor. Make sure that the codes assigned to each variable correspond exactly to the actual missing data codes used for that particular variable. In general, using the system-missing code '.' is best when importing data into HLM.

  • In the case of using ASCII data, the format specification may also be incorrect, causing HLM to read variables from the wrong columns in the data file.


The model should be respecified

Description

The model should be respecified. One (or more) of the random effects must be either deleted from the model or treated as fixed.

This happens when the iteration process forms an unusable (not positive-definite) tau where usually one of the diagonal elements has gone to effectively 0, or one or more of the correlations of tau falls outside –1 <= x <= 1. In practice, the latter is more likely.

Solution

If this happens, the output should give one some indication which variables are causing the trouble, and one of them should be fixed, or removed from the analysis entirely. Check the elements of the Tau matrices at all levels of the hierarchy. Small diagonal elements in these matrices indicate that negative variances may have been found, and that attempts by the program to fix this may have been unsuccessful. Intervention by the program in such cases is due to the use of the EM algorithm for estimation in HLM2/HLM3, and the unrestricted sections of HMLM/HMLM2. In the case of special models fitted using HMLM/HMLM2, (homogenous etc.), the Fisher accelerator is used and, if a negative variance is encountered, the program will exit with a message concerning the problem encountered during estimation. If this is the cause of the problem, you may want to fix the slope associated with the problem element of tau, in other words to assume that a common, fixed slope over units is adequate. To do this, remove the random coefficient from the slope in question.


There are no degrees of freedom to estimate sigma_squared

Description

The message

There are no degrees of freedom to estimate sigma_squared. Set sigma_squared to a constant, or delete one or more of the random effects from the model indicates that the model as specified implies the estimation of more parameters than can be done with available data. For example, if you have three data points nested within each level-2 unit and you allow two variables to vary randomly at level-2, the level-2 tau matrix will have three nonduplicated elements. Thus the number of random parameters at level-2 is equal to the number of data points/degrees of freedom and no "data remains" for the estimation of the remaining random parameter r at level-1.

Solution

In such a case, only one random effect can be accommodated at level-2. Either no random slope or, if the random slope is of specific interest, consider removing the random parameter associated with the intercept. Alternatively, compare model with a random intercept only to a model with only a random slope and use the model with the lowest deviance statistic.


There is a problem in the fixed part of the model

Description

There is a problem in the fixed portion of the model. A near singularity is likely. Possible sources are a collinearity or multicollinearity among the predictors. We suggest that you examine a correlation matrix among the fixed effect predictors.

This error message indicates that there are some of the fixed effects that have essentially the same relationship with the outcome variable. In order to perform iterations, the design matrix of predictors included in the analysis must be independent. In the example below, an example is given for a level-2 unit with 5 level-1 units nested within the level-2 unit. The first column represents the intercept term, which is by default included in any HLM model. The second column represents the scores of the 5 respondents from this level-2 unit. As the scores of all 5 respondents are very similar, the second column is almost a multiple of the first.

Intercept Score

1 20

1 20

1 20

1 20

1 21

Solution

This problem can be resolved in the following ways:

  • If retaining a variable that is a multiple of the intercept term is essential, the intercept term may be deleted from the model.

  • Use a correlation matrix of predictors within each higher level unit to find the pair or pairs of variables responsible for this problem. If, for example, a correlation close to 1 is observed for the predictors representing age and income, only one of the two predictors should be used in the model. Alternatively, a transformation of income could be considered in order to keep both variables in the model.


Unable to open file for writing

Description

When running an analysis, the message

unable to open c:\HLM\....out

may appear in the DOS window. This message indicates that the path specified for the output file in the Basic Model Specifications dialog box is invalid. Possible causes include an invalid path that does not exist, a path to a drive to which the user has no access, or a path to a full disk. For example, if the C: drive is write-protected, the user may not specify the output file to be created on this drive. Also, typos in the path name, for example C:\\HLM\...out (note the double \) will cause this warning to appear.

A similar message concerns the temporary files created during HLM analysis:

tmpfile: Permission denied

Unable to open temp file

Although a full disk may be the problem, this message is usually associated with write access problems. TMPFILE is the system routine that open temporary files, and these files will be opened in the root of the current drive.

Solution

Fix the typo in the path name, or make sure that this field points to a valid path and filename. When transferring files between computers, it may happen that the file directory as referred to in HLM is not present on another machine. If the problem is with read/write access, change the properties of the folder you are attempting to write too, or contact your system administrator and ask him to do this for you.


Very large numbers reported in output

Very large values printed to the output file, typically for the OLS regressions, are usually due to one of two causes:

  • If missing data codes, for example -999 are used in the data file and the user does not indicate that this value is a missing data code when making the SSM/MDM file, these values are regarded as true data values and all statistics for such a variable will be incorrect. This can be fixed by remaking the SSM/MDM file and taking care to indicate the presence of missing data in the SSM/MDM dialog box or prior to running the analysis, if this option was selected on the Make SSM/MDM dialog box. In the case of statistical package imports, the missing data code has to be specified as such in the data file prior to importing. Generally, using the system-missing code (typically ".") to indicate missing data in non-ASCII is advised.

  • Large differences in the scales of the variables included in the SSM/MDM file/analysis may also lead to very large starting values. Rescaling predictors prior to making the SSM/MDM file will solve the problem.


x of y units failed the conditioning check for inversion of the level-1 predictor matrix

Description

In order to perform iterations, the level-1 predictor matrix needs to be inverted. This message indicates that for a number of units a unique inverse for this matrix could not be found.

Inversion is dependent on the design matrix being of full rank. Columns of data (i.e. predictors included in the analysis) must be independent. In the example below, an example is given for a level-2 unit with 5 level-1 units nested within the level-2 unit. The first column represents the intercept term, which is by default included in any HLM model. The second column represents the scores of the 5 respondents from this level-2 unit. As the scores of all 5 respondents are very similar, the second column is almost a multiple of the first.

Intercept Score

1 20

1 20

1 20

1 20

1 21


Groups that passed the various checks on X'X (X = level-1 data), invertibility, positive determinant, condition number < 1E6, will have the OLS coefficients printed out. Groups that fail either of the first two checks will not appear, and will be counted as insufficient data. Groups that fail the condition check will be counted, and produce the message noted above. The level-1 coefficients for all "acceptable" units can be requested from the Basic Specifications menu by setting that field to the number of level-2 groups (or some large number). The results for "acceptable" groups will be printed. Note that this option is only available for HLM2.

This can also be caused by a lack of sufficient data for certain groups. For example, let’s say we have a model with 3 random effects. Groups with less than 3 level-1’s per level-2 (using HLM2 as an example) will cause this error, and probably at least some of the groups with three records will as well.
 

Solution

Possible solutions to this problem are:

  1. Scaling of the problem variable(s) may be considered, for example centering of predictors. Centering predictors reduce correlations between random effects.

  2. If retaining a variable that is a multiple of the intercept term is a problem, the intercept term may be deleted from the model.

  3. A mean value for a variable can be calculated at group level. The mean can then be used as a level-2 covariate instead.


Can't find what you looking for?
 
 

Back to Top