Differences

This shows you the differences between two versions of the page.

--- worked_example_prediction_train_and_test [2021/01/30 19:17] – [Training report] cloucera
+++ worked_example_prediction_train_and_test [2021/01/31 17:33] (current) – [Test report] cloucera
@@ Line 59: / Line 59: @@
 ===== Training report =====
 Once the launched study is finished, the report/results will be available in “My studies”.
 The report page of the Prediction-training tool includes different output results. You can download any table or image shown on the results page by clicking on the name right before it. You can also download the pathway and function matrices by clicking on //Circuit values//, For more information about each result please read [[prediction#training_report|Prediction - Training report]] and [[prediction#workflow|Prediction - Workflow]]  sections.
@@ Line 67: / Line 68: @@
 **Model Analysis**
-Hyperparameter search:
-{{ :svm.performance.heatmap.png?direct&400 | Hyperparameter search heatmap}}
 CV Performance:
+{{ ::model_stats.png?nolink |}}
 {{ :model_stats.txt| CV stats }}
@@ Line 126: / Line 125: @@
 PR curve over the test:
-{{ :split_test_pr.png?400 | Precision-recall (PR) curve for the test split. }}
+{{ :split_test_pr.png?400 | Precision-recall (PR) curve for the simulated test split. }}
 ROC curve over the test set:
@@ Line 132: / Line 131: @@
 {{ :split_test_roc.png?400 | ROC curve for the test split. }}
-Probability for the test set:
+Probability distributions of the positive class with respect to the true labels for the simulated test split:
 {{ :test_probability_boxplot.png?400 | ROC curve for the test split. }}
@@ Line 178: / Line 177: @@
 This is the most important result of our predictor, which is a matrix with three columns:
-  * Sample name: all the 125 samples in the used expression matrix file.
+  * Sample name: all the 125 samples (104 luminal A, 21 luminal B) in the used expression matrix file.
   * Prediction: the predicted group LumB (Luminal B) or LumA (Luminal A)
   * Probability LumB: this is the probability of being lumB, if it is 1 that means the predictor is 100% sure that the given result will be LumB.
 You can download the matrix of predicted experimental design by clicking on //Prediction results//.
 ===== Prediction evaluation =====
-Note that for this example we know beforehand the ground truth labels so we can compute the classification metrics as in the simulated split during the training phase. The ROC and PR curves are quite similar to those of the simulated split which inform us of the good generalization capabilities of the tool for this problem. The trend can also be observed from the companion metrics table.
+Note that for this example we know beforehand the ground truth labels so we can compute the classification metrics as in the simulated split during the training phase. The ROC and PR curves are quite similar to those of the simulated split which inform us of the good generalization capabilities of the tool for this problem. The trend can also be observed from the companion metrics table and the confusion matrix.
 {{ :split_prediction_roc.png?400 | ROC curve for the holdout split }}
@@ Line 193: / Line 193: @@
 {{ :split_prediction_probability_boxplot.png?400 | Probability distribution of th SVM for the holdout split }}
+^ statistic	^ value ^
+| Sensitivity	| 0.761904761904762 |
+| Specificity	| 0.913461538461538 |
+| Positive Predictive Value	| 0.64 |
+| Negative Predictive Value	| 0.95 |
+| False Positive Rate	| 0.0865384615384616 |
+| False Negative Rate	| 0.238095238095238 |
+| Likelihood Ratio Positive	| 8.8042328042328 |
+| Likelihood Ratio Negative	| 0.260651629072682 |
+| Percentage of data points in the main diagonal	| 0.888 |
+| Percentage of data points in the main diagonal corrected for agreement by chance	| 0.627659574468085 |
+| Rand index	| 0.799483870967742|
+| Rand index corrected for agreement by chance	| 0.525491509396793 |
+| Total Accuracy	| 0.888 |
-statistic	value
+^              ^^              Reference              ^^
+^              ^             ^    Lum A    ^    LumB    ^
-Sensitivity	0.761904761904762
+^    Prediction    ^    LumA    |    95    |    5    |
+^    :::    ^    LumB    |    9    |    16    |
-Specificity	0.913461538461538
-Positive Predictive Value	0.64
-Negative Predictive Value	0.95
-False Positive Rate	0.0865384615384616
-False Negative Rate	0.238095238095238
-Likelihood Ratio Positive	8.8042328042328
-Likelihood Ratio Negative	0.260651629072682
-Percentage of data points in the main diagonal	0.888
-Percentage of data points in the main diagonal corrected for agreement by chance	0.627659574468085
-Rand index	0.799483870967742
-Rand index corrected for agreement by chance	0.525491509396793
-Total Accuracy	0.888