Formulating Testable Trading Hypotheses

Good to see you again. Last lesson separated economic trading results from mechanical changes in market data: stock splits, dividends, futures rolls, and continuous series conventions. That distinction is a prerequisite for research design. A hypothesis is only meaningful if its signal is computed from data that would have existed at the decision time, and if its simulated return corresponds to instruments that could actually have been traded.

This lesson turns a market intuition into a proposition that historical evidence can genuinely contradict. By the end, you should be able to write a compact research specification containing an exact signal, decision time, holding period, benchmark, and precommitted rejection rule—before opening an equity-curve chart.

---

A trading idea is not yet a hypothesis

Consider these common statements:

• “Markets tend to rebound after selloffs.”
• “Trends persist.”
• “Companies with strong fundamentals outperform.”
• “Machine learning can find nonlinear patterns.”

They may be useful ideas, but none is a testable trading hypothesis. Each leaves enough discretion that, after examining data, a researcher could choose the lookback, threshold, asset universe, entry time, exit time, and evaluation metric that happens to look best.

A falsifiable trading hypothesis makes a narrow claim:

Under stated market conditions, information available at a stated time produces a specified trading decision whose net performance should exceed a stated alternative by a stated amount.

The claim may fail. Indeed, it must be possible for it to fail. If a poor result can always be explained away by changing a parameter, changing the sample, excluding an inconvenient period, or choosing a different benchmark, then the original “hypothesis” was not testable.

A useful separation is:

Layer	Question it answers
Economic rationale	Why might other market participants create this pattern?
Signal	What observable quantity represents the alleged pattern?
Trading rule	When and how does the signal create a position?
Evaluation rule	What result would count as support or rejection?

For example, “short-term selling pressure can temporarily push a broad equity ETF below fundamental value” is a rationale. It becomes a signal only once “short-term,” “selling pressure,” and “temporarily” have numerical definitions.

---

The five commitments that make a hypothesis testable

A robust first research specification should make five commitments. In practice, add the universe, costs, data source, and risk limits too—but these five establish the scientific core.

1. Define the signal and its information set

A signal must specify:

• the exact inputs and formula;
• the timestamp at which every input becomes available;
• the threshold or ranking rule;
• what happens when the signal is absent or undefined.

Suppose the idea is short-horizon reversal in a liquid equity ETF. Let $P_t^{\text{close}}$ be the closing price observed on day $t$. Define the preceding five-day close-to-close return as:

$$r_{t,5}^{CC}=\frac{P_t^{\text{close}}}{P_{t-5}^{\text{close}}}-1$$

A precise binary signal could be:

$$e_t=\{\begin{array}{ll}1,& r_{t,5}^{CC}\le -0.03\\ 0,& \text{otherwise}\end{array}$$

This rule says: after the close on day $t$, the strategy wants long exposure only if the ETF has fallen at least $3\%$ over the previous five closes.

The number $3\%$ is not sacred. The important discipline is that it is fixed before examining the test result. If you decide to evaluate $-2\%$, $-3\%$, and $-4\%$, record those as three planned candidates—not as one strategy with a conveniently selected threshold.

The information clock also matters. A closing price is only safely known after the market has closed. Therefore, a signal calculated from the close on day $t$ cannot honestly assume execution at that same close unless your data and execution model specifically support it. A conservative daily implementation submits the order for the next tradable opportunity.

2. Define the holding period

A signal without a holding period is an incomplete claim. “Buy after a selloff” may mean:

• buy at the next open and hold for one day;
• buy at the next open and hold for five trading days;
• buy and exit when the return becomes positive;
• buy until a contrary signal occurs.

Those are distinct strategies, with different turnover, cost sensitivity, market exposure, and statistical dependence.

For the example, state:

When $e_t=1$, buy at the open on day $t+1$, hold through the next trading day, and sell at the open on day $t+2$. When $e_t=0$, hold cash.

The gross active-trade return is then:

$$r_{t+1:t+2}^{OO}=\frac{P_{t+2}^{\text{open}}}{P_{t+1}^{\text{open}}}-1$$

The superscript $OO$ makes the entry and exit convention explicit: open to open. This is more than notation. A close-to-close backtest would answer a different question and may accidentally use unavailable closing information.

3. State the position rule and net-return convention

The signal is not yet the portfolio. In this simple example, the position is either fully long or entirely in cash:

$$w_{t+1:t+2}=e_t$$

Let $r_{t+1:t+2}^f$ be the return earned on uninvested cash during the holding interval, and let $c$ be the all-in round-trip cost of an active trade. Here, $c$ should include commissions and conservative estimates of entering at the ask and exiting at the bid.

The strategy’s net interval return is:

$$R_{t+1:t+2}=e_t\left(r_{t+1:t+2}^{OO}-c\right)+\left(1-e_t\right)r_{t+1:t+2}^f$$

This one expression rules out several common ambiguities:

• costs apply when the strategy trades, not on every day;
• idle capital earns the specified cash return;
• the position is long-only rather than implicitly short when no long signal appears;
• the decision is based on information from day $t$, not on day $t+1$.

Later modules will make the cost model and fill simulation more realistic. At this stage, state a conservative, auditable assumption rather than silently treating trading as free.

4. Choose a benchmark that answers the right question

A benchmark is a competing rule, not merely a well-known index pasted onto a chart.

For a long-only, intermittently invested ETF strategy, cash is a natural primary benchmark: does taking these conditional trades beat simply remaining uninvested after costs? But cash alone does not reveal whether the apparent performance is merely compensation for bearing equity-market risk.

Use two comparators:

1. Primary benchmark: cash.
   The strategy must earn a positive net excess return relative to the stated cash convention.

2. Risk-context comparator: a passive ETF allocation.
   Compare against a precommitted fixed allocation, such as $25\%$ in the ETF and $75\%$ in cash. This asks whether the conditional timing rule improves on a simple, lower-exposure equity allocation.

A passive comparator with fixed ETF weight $w_B=0.25$ has interval return:

$$B_{t+1:t+2}=0.25r_{t+1:t+2}^{OO}+0.75r_{t+1:t+2}^f$$

This is not a perfect replication of the strategy’s changing market exposure; no single benchmark can be. It is a deliberately simple alternative. Record the strategy’s realized average exposure and market beta alongside it, so that performance is not attributed to alpha when it is mostly explained by market direction.

The performance dashboard illustrates this point. A cumulative-return line may look encouraging while the rolling Sharpe ratio deteriorates or while beta changes substantially. Those rolling diagnostics are not proof of a hypothesis, but they help distinguish “the strategy made money” from “the strategy produced a repeatable return pattern distinct from its benchmark.”

5. Precommit a rejection criterion

A rejection criterion states what would make you stop believing this specific implementation deserves further development.

For the ETF reversal example, a research card might say:

Primary claim: A long-only one-day reversal rule, after modeled costs, earns economically meaningful returns above cash.

Benchmark claim: Its risk-adjusted performance is better than a fixed $25\%$ passive ETF allocation.

Rejection rule: In the locked evaluation period, reject this formulation if either:

1. mean net active-trade excess return is no greater than $2$ basis points per trade; or
2. the strategy’s net Sharpe ratio fails to exceed the passive comparator’s Sharpe ratio by at least $0.20$.

A basis point is $0.01\%$, so $2$ basis points is $0.02\%$. The threshold is deliberately economic, not just cosmetic: a tiny gross edge that disappears under plausible costs is not useful for an actively traded strategy.

These particular thresholds are illustrative. A suitable criterion depends on turnover, leverage, capacity, and the quality of the data. The principle is fixed:

Define the bar before seeing the final evidence, and let failure reject the defined version of the idea.

A failed reversal rule does not prove that every reversal effect is false. It means this universe, signal, timing convention, cost model, and holding rule did not meet the stated standard. A different holding period or a different universe is a new hypothesis that must be logged and tested as such.

---

A compact hypothesis card

Before coding, write a one-page research card. It should be specific enough that another person could implement the same test without asking what you “really meant.”

Field	Example specification
Claim	Large five-day declines in a liquid equity ETF are followed by positive short-horizon reversal.
Economic rationale	Temporary selling pressure or liquidity demand may overshoot fundamental repricing.
Instrument and universe	One explicitly named, highly liquid broad-market ETF; use the historically valid instrument record.
Decision time	Immediately after the official close of day $t$.
Signal	Go long if five-day close-to-close return is at most $-3\%$; otherwise hold cash.
Entry and exit	Enter at open $t+1$; exit at open $t+2$.
Position size	$100\%$ of allocated strategy capital when active; no leverage.
Costs	A stated commission plus conservative bid-ask spread assumptions on both sides.
Primary benchmark	Cash return over the same intervals.
Secondary comparator	Fixed $25\%$ passive ETF and $75\%$ cash allocation.
Evaluation period	A date range fixed before analysis, with a separately locked final period.
Rejection rule	Minimum net excess return per active trade and minimum risk-adjusted improvement over comparator.
Research log	Record every candidate rule, parameter range, rerun, data revision, and decision.

Notice what this card excludes: “optimize the lookback,” “find the best threshold,” and “adjust when performance is weak.” Those may be legitimate development tasks, but they introduce multiple testing and must be governed by a predeclared search procedure and a later validation process.

---

Why “good-looking” is not a research result

If you inspect hundreds of charts, rules, and parameter combinations, some will look impressive by chance. The more freedom you allow after observing results, the weaker the apparent evidence becomes.

A disciplined workflow therefore separates two activities:

Activity	Appropriate behavior
Exploration	Generate ideas, inspect data, learn market mechanics, and log all trials.
Confirmation	Freeze the candidate rule, its data treatment, benchmark, costs, and rejection criteria before evaluating it.

The key phrase is freeze the candidate. You may improve it later, but once you alter its threshold, holding period, universe, execution convention, or objective, you are no longer testing the same strategy.

The statistical machinery for assessing whether a result is distinguishable from noise comes next in the course. For now, do not confuse a rejection rule with a statistical test. The former is a precommitted decision standard; the latter quantifies uncertainty around observed performance. A serious strategy needs both.

---

Key takeaways

A trading intuition becomes falsifiable only when it commits to observable details. Define the signal formula and information timestamp, map that signal to a position, specify entry and exit times, include a net-return convention, and compare against a benchmark that addresses both opportunity cost and risk exposure.

Most importantly, state in advance what outcome would reject the specified strategy. A failed test is productive evidence when the hypothesis was narrow enough to fail honestly. Changing the rule after failure creates a new candidate, not a validation of the original one.

You have now completed the research-design foundation of this first module. Next, the course begins applied probability with conditional probability and Bayes’ rule: the framework for updating a trading hypothesis when new evidence arrives.