Validating quantitative data model
Otherwise we'd essentially have an external distribution that we would 'adjust' to 'better fit' a distribution that we don't know. What you might do is remove data points that are not relevant example losses from structured products if you don't have those in your portfolio.
So you're suggesting to make adjustments based on qualitative characteristics of your portfolio, rather than quantitative.
Cross-validation is the process of assessing how the results of a statistical analysis will generalize to an independent data set.
If the model has been estimated over some, but not all, of the available data, then the model using the estimated parameters can be used to predict the held-back data.
Mathematically, the definition of the residual for the i observation in the data set.
If the model fit to the data were correct, the residuals would approximate the random errors that make the relationship between the explanatory variables and the response variable a statistical relationship.
Serial correlation of the residuals can indicate model misspecification, and can be checked for with the Durbin–Watson statistic.
In contrast, the methods for validating diagnostic tests have not been agreed upon by the scientific community (Delong and Delong 1988).
The next section details the types of plots to use to test different aspects of a model and gives the correct interpretations of different results that could be observed for each type of plot.
A basic, though not quantitatively precise, way to check for problems that render a model inadequate is to conduct a visual examination of the residuals (the mispredictions of the data used in quantifying the model) to look for obvious deviations from randomness.
How would you approach model validation for an expert judgment model in the absence of default/loss history?
What type of testing can be performed when there are no "high credit risk"(e.g. Thank you, Related question on developing a credit scoring model: Expert System for Credit Scoring Related question on model validation criteria: Model Validation Criteria EDIT: The scope of validation I have been able to come up with is (for the most part) qualitative.