Deploying MCMC Outside of Bayesian Frameworks

At face value, running an MCMC against a database of your own model’s instances looks like the most efficient way to overfit ever conceived. On paper, you are feeding a sampler the exact cases your model was built to capture, and asking it to estimate the relative contribution of each parameter.

Yet this is exactly what I do, and the distinction is epistemic. The protection against overfitting is procedural. The sampler is allowed to propose relationships, but it is not allowed to justify them. Any relationship it identifies must survive an entirely separate line of reasoning before it ever earns a place in the model. A parameter is not added because it improves fit. It is added only when the relationship can be given a coherent causal explanation within market structure, whether it refines an existing mechanism or introduces one that was previously unmodeled.

This creates an asymmetry in which statistical evidence can guide attention, but it cannot establish causation. The reason I run it anyway is simple: the market may be expressing information that the current parameterization fails to capture. If statistically observable relationships correspond to plausible market mechanisms, I want to find them. I do not treat parameters as truths discovered in data. They exist because they were defined before any instance was ever observed. The model is then evaluated through direct exposure to real instances, either through historical backtesting or forward testing. The MCMC does not validate the model. It interrogates it, identifying relationships the current structure may not fully account for and generating candidates for investigation.

Crucially, the analysis is not restricted to the model’s internal variables. A thirty-parameter model may be evaluated against seventy-five observed features. The purpose is not parameter discovery or backtest optimization, but the identification of relationships that warrant structural attention. Most of these are discarded. Some are statistical artifacts. Some are noise. Some are sample-specific. Some collapse under structural inspection. Occasionally, the sampler isolates a relationship that is both statistically persistent and structurally intelligible. These are the only cases that matter.

A relationship does not earn consideration merely because a plausible explanation can be attached to it after the fact. The explanation must integrate coherently with the existing framework and generate implications beyond the observation that motivated it. It must clarify not only why the relationship appears, but also the conditions under which it should weaken, disappear, or fail. Relationships that survive that standard may indicate misspecification of an existing parameter, an omitted mechanism, or a structural feature the model does not currently account for. Statistical persistence alone is not meaningful. A repeated pattern is only an observation until it has a reason to exist. Likewise, a compelling explanation without empirical support is a story. Neither is sufficient on its own.

For me, what others would call a small sample is actually often enough. Not because it proves universality, the goal is not universality. It is to surface relationships worth examining. If cumulative volume at entry consistently precedes larger outcomes, and there is a coherent mechanism explaining why, given the framework, additional observation does not add clarity before structural evaluation begins. The real test comes after: does the mechanism remain coherent under scrutiny, across regime shifts, and across both wins and failures. That matters more than frequency in any fixed sample.

Conversely, ten years of data showing consistent short-term mean reversion after sharp one-sided dislocations on a specific asset tells me very little without a structural explanation for why that effect should exist. Frequency alone is repetition. It describes what happened often, not what produced it, when it fails, or under what conditions it breaks.

That is “built on structural reasoning rather than statistical discovery” in action.