Bayesian PCA

By | January 5, 2015

Authors: Jan Smycka, Petr Keil

This post introduces experimental R package bPCA which we developed with Jan Smycka, who actually came with the idea. We do not guarantee the very idea to be correct and there certainly are bugs – we invite anyone to show us wrong, or to contribute.

Rationale of bPCA

Here is a summary of classical (‘non-Bayesian’) PCA from Wikipedia:

PCA can be thought of as fitting an n-dimensional ellipsoid to the data, where each axis of the ellipsoid represents a principal component. If some axis of the ellipse is small, then the variance along that axis is also small, and by omitting that axis and its corresponding principal component from our representation of the dataset, we lose only a commensurately small amount of information.

To find the axes of the ellipse, we must first subtract the mean of each variable from the dataset to center the data around the origin. Then, we compute the covariance matrix of the data, and calculate the eigenvalues and corresponding eigenvectors of this covariance matrix. Then, we must orthogonalize the set of eigenvectors, and normalize each to become unit vectors. Once this is done, each of the mutually orthogonal, unit eigenvectors can be interpreted as an axis of the ellipsoid fitted to the data.

The bold terms above should evoke Multivariate Normal (MVN) distribution. Jan’s idea was that we can think about the data as being MVN-distributed. If we make such assumption, we can fit the MVN to the data using MCMC, get posterior distributions of means and covariances, and subsequently also posterior distributions of PCA loadings, scores and eigenvalues.

Potential advantages

  • Prior information about associations between variables can be provided (through covariance matrix).
  • We can assess stability (uncertainty) of a PCA, especially when only small sample sizes are available.
  • The posterior distributions for a PCA scores and loadings can be extracted for further use, e.g. anywhere where propagation of uncertainty is of interest.

Potential limitations

  • The algorithm is extremely slow for data with many variables (over 100)
  • The assumption of the underlying MVN can pose a problem.
  • In the summary and plotting functions we may be breaking some important ties between values in the MCMC chains. The cleanest solution (which we avoid) would perhaps be to calculate the eigenvalues, loadings and scores all in the JAGS sampler.

Installing the bPCA package

These two commands will install the package from the GitHub repository directly to your computer:

library(devtools)
install_github("bPCA", username="petrkeil")

To load the package:

library(bPCA)

Note: bPCA depends on R2jags, MASS, Matrix, coda, reshape, lattice.

Example use of bPCA package

Here we apply bPCA to the first four columns in the built-in dataset iris.

  summary(iris[,1:4])
##   Sepal.Length   Sepal.Width    Petal.Length   Petal.Width 
##  Min.   :4.30   Min.   :2.00   Min.   :1.00   Min.   :0.1  
##  1st Qu.:5.10   1st Qu.:2.80   1st Qu.:1.60   1st Qu.:0.3  
##  Median :5.80   Median :3.00   Median :4.35   Median :1.3  
##  Mean   :5.84   Mean   :3.06   Mean   :3.76   Mean   :1.2  
##  3rd Qu.:6.40   3rd Qu.:3.30   3rd Qu.:5.10   3rd Qu.:1.8  
##  Max.   :7.90   Max.   :4.40   Max.   :6.90   Max.   :2.5

We fit the MVN distribution to the data using sim.bPCA function. This function is esentially a wrapper around a piece of JAGS code. Each value of mean and covariance is represented by MCMC chain. Note: The function sim.bPCA also provides an option to supply prior covariance matrix.

So here we go:

bPCA.fitted <- sim.bPCA(iris[,1:4], n.chains=3, n.iter=1000, n.burnin=500)
## Compiling model graph
##    Resolving undeclared variables
##    Allocating nodes
##    Graph Size: 195
## 
## Initializing model

We can extract posterior distributions of eigenvalues and percentages of explained variance, and show them as boxplots:

eigenvalplots.bPCA(bPCA.fitted, iris[,1:4])

plot of chunk unnamed-chunk-5

## $Eigenvalues
##        V1              V2              V3               V4        
##  Min.   : 2.98   Min.   :0.166   Min.   :0.0495   Min.   :0.0169  
##  1st Qu.: 3.93   1st Qu.:0.222   1st Qu.:0.0711   1st Qu.:0.0214  
##  Median : 4.25   Median :0.240   Median :0.0765   Median :0.0231  
##  Mean   : 4.40   Mean   :0.247   Mean   :0.0773   Mean   :0.0233  
##  3rd Qu.: 4.61   3rd Qu.:0.261   3rd Qu.:0.0828   3rd Qu.:0.0249  
##  Max.   :71.65   Max.   :2.521   Max.   :0.1158   Max.   :0.0336  
## 
## $Exp.var
##        V1             V2             V3              V4        
##  Min.   :88.6   Min.   :2.90   Min.   :0.103   Min.   :0.0335  
##  1st Qu.:91.9   1st Qu.:4.73   1st Qu.:1.501   1st Qu.:0.4513  
##  Median :92.5   Median :5.24   Median :1.672   Median :0.5065  
##  Mean   :92.5   Mean   :5.30   Mean   :1.688   Mean   :0.5091  
##  3rd Qu.:93.2   3rd Qu.:5.81   3rd Qu.:1.855   3rd Qu.:0.5596  
##  Max.   :96.9   Max.   :8.93   Max.   :2.760   Max.   :0.8391

We can see that the boxplots are pretty narrow, especially for the explained variability, indicating stability and robustness (low uncertainty) of the results, which comes with the large sample size of the iris dataset.

Now it is getting a bit wild: We can make a PCA biplot for 5%, 50% and 97.5% quantiles of the posterior distributions:

biplots.bPCA(bPCA.fitted, iris[,1:4], axes.to.plot=1:2, scale=0.1)

plot of chunk unnamed-chunk-6 You can try to play around with the scale parameter, which controls the relative width of the plotted arrows.

We can also extract MCMC chains (posterior distributions) for PCA loadings from the bPCA.fitted object and summarize them by quantiles (5%, 50% and 97.5%):

loadings.chain <- get.loadings.chain.bPCA(bPCA.fitted, iris[,1:4])

And we can summarize the posterior distributions of the PCA loadings by quantiles (5%, 50% and 97.5%):

summary.loadings.bPCA(loadings.chain, vars.to.get=1:4, axes.to.get=1:2)

plot of chunk unnamed-chunk-8

## $`0.025`
##               Comp.1  Comp.2
## Sepal.Length -0.3778 -0.7076
## Sepal.Width  -0.1170 -0.8034
## Petal.Length -0.8538 -0.2152
## Petal.Width  -0.3600 -0.1539
## 
## $`0.5`
##                Comp.1   Comp.2
## Sepal.Length  0.35944 -0.61570
## Sepal.Width  -0.08496 -0.70617
## Petal.Length  0.85619  0.15163
## Petal.Width   0.35708  0.03422
## 
## $`0.975`
##               Comp.1 Comp.2
## Sepal.Length 0.38875 0.7299
## Sepal.Width  0.06304 0.7700
## Petal.Length 0.86535 0.2088
## Petal.Width  0.37403 0.1411

And finally, here we extract and summarize (by quantiles) the MCMC chains for PCA scores from the bPCA.fitted object:

scores.chain <- get.scores.chain.bPCA(bPCA.fitted, iris[,1:4])
summary.scores.bPCA(scores.chain, axes.to.get=1:2)
## $`0.025`
##      Comp.1   Comp.2
## 1   -2.9945 -0.42636
## 2   -3.0220 -0.26703
## 3   -3.1968 -0.24904
## 4   -3.0526 -0.41919
## 5   -3.0390 -0.43360
## 6   -2.5890 -0.83825
## 7   -3.1278 -0.19983
## 8   -2.9350 -0.26937
## 9   -3.1923 -0.68180
## 10  -2.9799 -0.20667
## 11  -2.8150 -0.74704
## 12  -2.9195 -0.13083
## 13  -3.0930 -0.32803
## 14  -3.5298 -0.62226
## 15  -2.9551 -1.28148
## 16  -2.6988 -1.43877
## 17  -2.9336 -0.91308
## 18  -2.9587 -0.41733
## 19  -2.5089 -0.96688
## 20  -2.8954 -0.61933
## 21  -2.6200 -0.48977
## 22  -2.8515 -0.53664
## 23  -3.5246 -0.26053
## 24  -2.6117 -0.19862
## 25  -2.6615 -0.13543
## 26  -2.8144 -0.23504
## 27  -2.7785 -0.23201
## 28  -2.8721 -0.47107
## 29  -2.9495 -0.41841
## 30  -2.9388 -0.29616
## 31  -2.8948 -0.30064
## 32  -2.7204 -0.51132
## 33  -2.9557 -0.92059
## 34  -2.9089 -1.19445
## 35  -2.9444 -0.21309
## 36  -3.1756 -0.18107
## 37  -2.9330 -0.70611
## 38  -3.1107 -0.37718
## 39  -3.2867 -0.59402
## 40  -2.8992 -0.33360
## 41  -3.0811 -0.37141
## 42  -3.1590 -1.03161
## 43  -3.3032 -0.45204
## 44  -2.7161 -0.29379
## 45  -2.5167 -0.53797
## 46  -3.0221 -0.34264
## 47  -2.8453 -0.60550
## 48  -3.1465 -0.32943
## 49  -2.8514 -0.68201
## 50  -3.0119 -0.21535
## 51  -1.3544 -0.77324
## 52  -0.9839 -0.39528
## 53  -1.5238 -0.59575
## 54  -0.2482 -0.88436
## 55  -1.1261 -0.15818
## 56  -0.6631 -0.48704
## 57  -1.1463 -0.37089
## 58  -1.0575 -1.06439
## 59  -1.0924 -0.31648
## 60  -0.3286 -0.78917
## 61  -0.8199 -1.31909
## 62  -0.5461 -0.17291
## 63  -0.2857 -0.61527
## 64  -1.0173 -0.19642
## 65  -0.4827 -0.31457
## 66  -0.9862 -0.54663
## 67  -0.6861 -0.43422
## 68  -0.2783 -0.39260
## 69  -0.9584 -0.61611
## 70  -0.2919 -0.64001
## 71  -1.1538 -0.18503
## 72  -0.3963 -0.13155
## 73  -1.3208 -0.40556
## 74  -0.9519 -0.25286
## 75  -0.7605 -0.22630
## 76  -0.9509 -0.40839
## 77  -1.3793 -0.34063
## 78  -1.6020 -0.36191
## 79  -0.8444 -0.23378
## 80  -0.6128 -0.42286
## 81  -0.3763 -0.75971
## 82  -0.4977 -0.73371
## 83  -0.2528 -0.37217
## 84  -1.4004 -0.50287
## 85  -0.6105 -0.57284
## 86  -0.8563 -0.29098
## 87  -1.2754 -0.49006
## 88  -0.8350 -0.44652
## 89  -0.2866 -0.33572
## 90  -0.2512 -0.74019
## 91  -0.4798 -0.73345
## 92  -0.9275 -0.10595
## 93  -0.2729 -0.46296
## 94  -1.0123 -1.06837
## 95  -0.3746 -0.56662
## 96  -0.3625 -0.27651
## 97  -0.4028 -0.35773
## 98  -0.6848 -0.09396
## 99  -1.2136 -0.81660
## 100 -0.3258 -0.40892
## 101 -2.5713 -0.18013
## 102 -1.4324 -0.67011
## 103 -2.6626 -0.46756
## 104 -2.0020 -0.28095
## 105 -2.3867 -0.15590
## 106 -3.4503 -0.69482
## 107 -0.5311 -1.28247
## 108 -2.9790 -0.49010
## 109 -2.3466 -0.35346
## 110 -2.9888 -0.92759
## 111 -1.7069 -0.34849
## 112 -1.8304 -0.31074
## 113 -2.2086 -0.32870
## 114 -1.3618 -0.86831
## 115 -1.6072 -0.66105
## 116 -1.9468 -0.24438
## 117 -1.9890 -0.15275
## 118 -3.5840 -1.32729
## 119 -3.8369 -0.41337
## 120 -1.3120 -0.83754
## 121 -2.4772 -0.50360
## 122 -1.2186 -0.71013
## 123 -3.5489 -0.61123
## 124 -1.4162 -0.28498
## 125 -2.3237 -0.45995
## 126 -2.6697 -0.68501
## 127 -1.2880 -0.26053
## 128 -1.3251 -0.20711
## 129 -2.1520 -0.31536
## 130 -2.4393 -0.58100
## 131 -2.8874 -0.50626
## 132 -3.3338 -1.50967
## 133 -2.1877 -0.32492
## 134 -1.4748 -0.22420
## 135 -1.8005 -0.59277
## 136 -3.1378 -0.82452
## 137 -2.1887 -0.28358
## 138 -1.9458 -0.16154
## 139 -1.2017 -0.25808
## 140 -2.1556 -0.48238
## 141 -2.3572 -0.30983
## 142 -1.9740 -0.51809
## 143 -1.4324 -0.67011
## 144 -2.6091 -0.41233
## 145 -2.4668 -0.44260
## 146 -1.9861 -0.29961
## 147 -1.5468 -0.45892
## 148 -1.8046 -0.18531
## 149 -1.9449 -0.25447
## 150 -1.4185 -0.38366
## 
## $`0.5`
##        Comp.1    Comp.2
## 1   -2.671377 -0.259318
## 2   -2.699948  0.127838
## 3   -2.875261  0.091951
## 4   -2.732388  0.265896
## 5   -2.715839 -0.267358
## 6   -2.267067 -0.689012
## 7   -2.807008  0.038656
## 8   -2.612510 -0.104769
## 9   -2.871063  0.523541
## 10  -2.658185  0.066666
## 11  -2.492835 -0.587827
## 12  -2.599748 -0.002079
## 13  -2.772043  0.185107
## 14  -3.209090  0.452340
## 15  -2.629543 -1.115099
## 16  -2.371597 -1.283516
## 17  -2.609313 -0.751890
## 18  -2.635087 -0.252319
## 19  -2.184339 -0.817121
## 20  -2.574345 -0.458948
## 21  -2.296517 -0.332557
## 22  -2.530829 -0.378571
## 23  -3.202559 -0.070955
## 24  -2.289342 -0.045918
## 25  -2.342584  0.012187
## 26  -2.491869  0.099518
## 27  -2.455699 -0.075516
## 28  -2.549565 -0.309779
## 29  -2.626040 -0.251855
## 30  -2.618391  0.146485
## 31  -2.573522  0.156408
## 32  -2.396037 -0.351815
## 33  -2.633389 -0.757759
## 34  -2.584196 -1.035418
## 35  -2.622615  0.073934
## 36  -2.851514 -0.016561
## 37  -2.611149 -0.532321
## 38  -2.786797 -0.209118
## 39  -2.966101  0.432322
## 40  -2.575980 -0.170671
## 41  -2.756894 -0.202845
## 42  -2.831526  0.888092
## 43  -2.983116  0.283589
## 44  -2.392183 -0.135986
## 45  -2.196245 -0.388486
## 46  -2.700490  0.200079
## 47  -2.524641 -0.449335
## 48  -2.826260  0.173801
## 49  -2.529763 -0.521761
## 50  -2.689465 -0.048874
## 51   1.268302 -0.633451
## 52   0.915357 -0.280192
## 53   1.448554 -0.454749
## 54   0.171293  0.794701
## 55   1.074534 -0.030987
## 56   0.628135  0.378864
## 57   1.077573 -0.249134
## 58  -0.733562  0.968358
## 59   1.029810 -0.179587
## 60  -0.007162  0.680486
## 61  -0.491335  1.230741
## 62   0.497407  0.064422
## 63   0.253865  0.509018
## 64   0.971504  0.085857
## 65  -0.161857  0.219421
## 66   0.912530 -0.421342
## 67   0.645744  0.305669
## 68   0.223734  0.299880
## 69   0.932890  0.504053
## 70   0.042361  0.550371
## 71   1.101161  0.031869
## 72   0.343859  0.037803
## 73   1.284913  0.287485
## 74   0.908184  0.146444
## 75   0.700927 -0.104657
## 76   0.885841 -0.283379
## 77   1.318325 -0.189180
## 78   1.541962 -0.222255
## 79   0.799263  0.125325
## 80  -0.292353  0.334558
## 81  -0.058123  0.669880
## 82  -0.175684  0.646515
## 83   0.125068  0.281464
## 84   1.366494  0.377788
## 85   0.573948  0.431905
## 86   0.789565 -0.161653
## 87   1.204981 -0.362714
## 88   0.803343  0.331304
## 89   0.232008  0.228170
## 90   0.155279  0.646494
## 91   0.453024  0.633965
## 92   0.876799  0.009900
## 93   0.218582  0.370715
## 94  -0.687917  0.975420
## 95   0.344313  0.467736
## 96   0.318666  0.174674
## 97   0.362647  0.255025
## 98   0.628375 -0.001028
## 99  -0.891548  0.720374
## 100  0.285907  0.312411
## 101  2.513993 -0.017827
## 102  1.401073  0.525489
## 103  2.599619 -0.290050
## 104  1.957259  0.130347
## 105  2.335410  0.003607
## 106  3.379995 -0.480774
## 107  0.507080  1.137062
## 108  2.916433 -0.288510
## 109  2.307418  0.190578
## 110  2.897674 -0.730553
## 111  1.644758 -0.201914
## 112  1.789552  0.169510
## 113  2.149511 -0.171880
## 114  1.332655  0.727648
## 115  1.570282  0.475636
## 116  1.886920 -0.076666
## 117  1.935312 -0.018567
## 118  3.466451 -1.114330
## 119  3.780691 -0.178488
## 120  1.287930  0.719659
## 121  2.410029 -0.330961
## 122  1.184991  0.548114
## 123  3.484630 -0.383773
## 124  1.374922  0.162619
## 125  2.257178 -0.288434
## 126  2.596037 -0.502984
## 127  1.244322  0.138039
## 128  1.276839  0.070291
## 129  2.108444  0.157925
## 130  2.371822 -0.402584
## 131  2.827070 -0.305660
## 132  3.211227 -1.310769
## 133  2.144511  0.164440
## 134  1.430821  0.101343
## 135  1.767732  0.451554
## 136  3.059497 -0.620381
## 137  2.126724 -0.095723
## 138  1.889082 -0.025442
## 139  1.154406  0.117955
## 140  2.090047 -0.327417
## 141  2.296565 -0.136267
## 142  1.904761 -0.364884
## 143  1.401073  0.525489
## 144  2.545625 -0.228936
## 145  2.401245 -0.257039
## 146  1.928156 -0.144625
## 147  1.514905  0.333636
## 148  1.749335 -0.041428
## 149  1.883553 -0.075459
## 150  1.375362  0.230825
## 
## $`0.975`
##     Comp.1  Comp.2
## 1   2.7209 0.44289
## 2   2.7630 0.30172
## 3   2.9357 0.27139
## 4   2.7973 0.44087
## 5   2.7638 0.44578
## 6   2.3039 0.84233
## 7   2.8652 0.21497
## 8   2.6664 0.28450
## 9   2.9451 0.70411
## 10  2.7218 0.23763
## 11  2.5352 0.76290
## 12  2.6552 0.14467
## 13  2.8370 0.35944
## 14  3.2795 0.64695
## 15  2.6549 1.29832
## 16  2.3885 1.43890
## 17  2.6436 0.92330
## 18  2.6840 0.43231
## 19  2.2207 0.98106
## 20  2.6166 0.62115
## 21  2.3481 0.51324
## 22  2.5738 0.54063
## 23  3.2517 0.26344
## 24  2.3428 0.20624
## 25  2.4008 0.15423
## 26  2.5562 0.26648
## 27  2.5074 0.24240
## 28  2.5985 0.48893
## 29  2.6773 0.43585
## 30  2.6809 0.31586
## 31  2.6366 0.32441
## 32  2.4444 0.53355
## 33  2.6688 0.92609
## 34  2.6093 1.20232
## 35  2.6847 0.24085
## 36  2.9070 0.20396
## 37  2.6566 0.72740
## 38  2.8375 0.39013
## 39  3.0357 0.61786
## 40  2.6294 0.35106
## 41  2.8060 0.38525
## 42  2.9177 1.07874
## 43  3.0494 0.47405
## 44  2.4403 0.29198
## 45  2.2399 0.54073
## 46  2.7639 0.37436
## 47  2.5681 0.61354
## 48  2.8887 0.35392
## 49  2.5725 0.69657
## 50  2.7442 0.23392
## 51  1.6259 0.77721
## 52  1.2699 0.38337
## 53  1.8026 0.59511
## 54  0.5120 0.90289
## 55  1.4236 0.15780
## 56  0.9728 0.49476
## 57  1.4335 0.34394
## 58  0.8205 1.08165
## 59  1.3783 0.31694
## 60  0.3240 0.79577
## 61  0.5895 1.34399
## 62  0.8498 0.17806
## 63  0.5962 0.66153
## 64  1.3227 0.20592
## 65  0.2651 0.32794
## 66  1.2657 0.54676
## 67  0.9975 0.43328
## 68  0.5748 0.41577
## 69  1.2721 0.63917
## 70  0.3759 0.66086
## 71  1.4552 0.17508
## 72  0.6934 0.14742
## 73  1.6267 0.42088
## 74  1.2575 0.26630
## 75  1.0498 0.22889
## 76  1.2360 0.40861
## 77  1.6658 0.34556
## 78  1.8936 0.35100
## 79  1.1518 0.23905
## 80  0.3614 0.45265
## 81  0.2846 0.78422
## 82  0.2713 0.76330
## 83  0.4751 0.39045
## 84  1.7108 0.50471
## 85  0.9241 0.56841
## 86  1.1480 0.24912
## 87  1.5581 0.48878
## 88  1.1443 0.48011
## 89  0.5863 0.34242
## 90  0.4964 0.75552
## 91  0.7947 0.74466
## 92  1.2280 0.11246
## 93  0.5684 0.48309
## 94  0.7767 1.08867
## 95  0.6907 0.57685
## 96  0.6710 0.28806
## 97  0.7150 0.36698
## 98  0.9787 0.09422
## 99  0.9692 0.83704
## 100 0.6379 0.42038
## 101 2.8701 0.15128
## 102 1.7488 0.66326
## 103 2.9513 0.45023
## 104 2.3069 0.27417
## 105 2.6861 0.14832
## 106 3.7321 0.68620
## 107 0.8508 1.27931
## 108 3.2659 0.48569
## 109 2.6512 0.35755
## 110 3.2577 0.87889
## 111 2.0014 0.31212
## 112 2.1367 0.30647
## 113 2.5004 0.30524
## 114 1.6769 0.86390
## 115 1.9210 0.64564
## 116 2.2409 0.19808
## 117 2.2829 0.12614
## 118 3.8236 1.29230
## 119 4.1255 0.41425
## 120 1.6293 0.85344
## 121 2.7664 0.46661
## 122 1.5331 0.69862
## 123 3.8341 0.61355
## 124 1.7216 0.28982
## 125 2.6141 0.41909
## 126 2.9529 0.66579
## 127 1.5925 0.25969
## 128 1.6272 0.20080
## 129 2.4590 0.30765
## 130 2.7253 0.58384
## 131 3.1748 0.50864
## 132 3.5691 1.48892
## 133 2.4953 0.31995
## 134 1.7774 0.23394
## 135 2.1071 0.60132
## 136 3.4170 0.81262
## 137 2.4814 0.22744
## 138 2.2398 0.12679
## 139 1.5062 0.24873
## 140 2.4452 0.46039
## 141 2.6505 0.27505
## 142 2.2603 0.48980
## 143 1.7488 0.66326
## 144 2.9007 0.36825
## 145 2.7576 0.39275
## 146 2.2785 0.26892
## 147 1.8586 0.46212
## 148 2.0979 0.15503
## 149 2.2382 0.19798
## 150 1.7287 0.37263

6 thoughts on “Bayesian PCA

  1. Robert Grant

    Great work! I've been thinking about Bayesian dimension reduction recently because of its added flexibility. I think it could be very useful. But on the speed problem: the parameters are inherently highly correlated, so Gibbs sampling explores that hyper-cigar shape very slowly. Try RStan instead - you won't regret it!

    Reply
  2. Pingback: Weekly links round-up: 06/02/2015 | BES Quantitative Ecology Blog

  3. Pingback: » Bayesian Biostatistics, 25-27 Jan 2016, iDiv, Germany

  4. Brandon

    Is this package still available? The installation won't work. At first it said to use the format install_github("petrkeil/bPCA"), but that doesn't work either. It seems it is not available on my end.

    Reply
    1. Petr Keil Post author

      Hi Brandon, it is still working with me, but I am on Linux (so there may be problems on Windows or Mac). In any case, you need to have JAGS installed (not an R package, it's a stand-alone software), and bPCA also depends on 'rjags' package. So check the dependencies.

      Reply
  5. Pingback: Introduction to maximum likelihood and Bayesian statistics for ecologists (1-3 March 2017, iDiv) – Petr Keil

Leave a Reply

Your email address will not be published. Required fields are marked *