SINFA: Multivariate uncertainty analysis for confusion matrices

Abstract

Stimulus-response confusion matrices are often used to summarize the results of perceptual experiments in which subjects must identify which stimulus, from a closed set of stimuli, has been presented on a given trial. The subject's performance is often assessed by calculating the amount of stimulus information transmitted. In the terminology of uncertainty analysis (Garner, 1962), transmitted information is a special case of contingent uncertainty U(S:R), where the contingency is between stimuli and responses. When the stimuli are multidimensional, it is possible to assess the amount of information transmitted for a single dimension or feature. This is accomplished by combining the cells of the confusion matrix to form a feature confusion matrix and calculating contingent uncertainty for this reduced matrix. If stimulus features are orthogonal and if the subject's responses to the different features are independent, then U(S:R) is simply the sum of the contingent uncertainties for the features. If the stimulus features are redundant or if the responses to the features are not independent, then the sum of the feature contingent uncertainties will be greater than U(S: R) because the redundancy has not been taken into account. For multidimensional stimulus sets containing features FI , F2, ... , Fk, U(S:R) may be represented as a multiple contingent uncertainty U(FI , F2, ... , Fk:h,f2, ... , fk), where uppercase letters designate stimulus features and lowercase letters designate response features. Garner (1962) has shown that the multiple contingent uncertainty can be partitioned into additive components and that there are a great many ways to perform the partitioning. One method of partitioning involves calculation of k2 terms, k of which are of the form: U(FI :(1), UF1 f1 (F2:f2), UFIF,flf,(Fa:fa), ... ,UF1F, ... Fk-lflf, ... fk-l (Fk : fk). The first term is the contingent uncertainty for Fl. The second term is a partial contingent uncer- tainty for F2 in which the effects of F1 have been partialed out. The third term is a partial contingent uncertainty for Fa in which the effects of both FI and F2 have been partialed. The sum of k such terms will always be less than or equal to U(S: R) and, therefore, the sum gives a measure of the amount of transmit- ted information accounted for by a given set of features. The sum is almost always less than U(S: R) because the complete partitioning also involves k(k - 1 ) additional terms whic.h represent partial "cross-contingencies" between a stimulus feature and a different response feature. These quantities are analogous to dimensional interactions. SINFA (sequential information analysis) is a FORTRAN IV program which partitions U(S: R) into a set of contingent uncertainties, each of which is associated with a single feature. Cross-contingencies are not computed. The order in which feature contingent uncertainties are calculated may be specified by the user (fixed mode) or may be determined by the program (free mode). In free mode, contingent uncertainty for a feature in a given iteration is divided by the maximum possible value for that feature to obtain a relative performance measure. In a given iteration, the feature with the highest relative level of performance is selected. In free mode an additional criterion is imposed on feature selection. A feature will not be selected unless the partial contingent' necessary for the feature, divided by U(S: R), is at least C, where C may be specified by the user. A more detailed description and illustration of the analysis may be found in Wang and Bilger (1973), and application of the method to con- fusion matrices for individual subjects is reported in Bilgerand Wang(1976). Input. Required input to the program consists of the number of stimuli, the number of features, a set of feature names and feature definitions, the order in which features are to be selected if the program is to be run in fixed mode, and the stimulus-response confusion matrix. Optional input consists of a 75-character title, a user- specified value of C (default value is .01), and a request for optional output described below. Output. The SINFA summary table provides the following information for each iteration: the name of the feature selected, the partial contingent uncertainty for that feature, and the partial contingent uncertainty divided by U(S: R). The latter value must be at least equal to C, and it gives the proportion of the total transmitted information accounted for by the feature when previously selected features are held constant. The sum across iterations for each of these measures is also given. Optional output consists of a table for each iteration which gives the following information for each feature: the partial contingent uncertainty, the maximum value of the partial contingent uncertainty, and the ratio of these two quantities (i.e., the relative performance measure). Program limitations. SINFA will accept a confusion matrix for 30 stimuli and will accept up to 20 features with a maximum of 8 categories each. These values may easily be modified by changing the appropriate dimension statements. The program was written for a UNIVAC 1108 computer and requires 12,000 locations. Availability. Detailed instructions for running the program and a source listing are available without cost from the author at the Department of Psychology, University of Maryland Baltimore County, Baltimore, Maryland 21228. REFERENCES BILGER. R. C., & W ANG. M. D. Consonant confusions in patients with sensorineural hearing loss. Journal of Speech and Hearing Research. 1976. 19. in press. GARNER. W. R. Uncertainty and structure as psychological concepts. New York: Wiley, 1962. WANG. M. D.. & BILGER. R. C. Consonant confusions in noise: A study of perceptual features. Journal of the Acoustical Society of America, 1973, 54. 1248-1266. (Received for publication July 27,1976. Accepted July 27.1976.) 刺激-応答混乱マトリックスは、しばしば、被検者がどの刺激が、刺激の閉集合から、所定の試行の上で示されたかについて確認するにちがいない知覚的な実験の結果をまとめるために利用される。 被検者のパフォーマンスは、しばしば、情報が送った刺激の量を計算することによって評価される。 不確実性解析（Garner、1962）の用語では、伝達情報量は付随的な不確実性U（S：R）の特別なケースである、そこで、偶発は刺激と応答の間にある。 刺激が多次元的であるとき、一つの次元または特徴のために送られる情報量を評価することができる。 これは、特徴コンフュージョンマトリックスを作るためにコンフュージョンマトリックスのセルを結合して、この減少したマトリックスのために付随的な不確実性を計算することによって達成される。 刺激が大きな位置を占めるかどうかは直交である、そして、異なる特徴への被検者の応答が独立しているならば、U（S：R）は単に特徴のための付随的な不確実性の金額である。 刺激が大きな位置を占めるかどうかは不要である、あるいは、特徴への応答が独立していないならば、冗長性が考慮されなかったので、特徴偶発的な不確実性の和はU（S：R）より大きい。