|
Optimize mode is a special run mode that performs multiple back-to-back tests. Despite the name, it is not only for parameter optimization — it is the right mode for any task that requires more than a single test, including:
•exploring a parameter space (the traditional optimization use case)
•running the same script across multiple date ranges
•generating a walk-forward test
•repeating a randomized test many times to study the distribution of outcomes
•splitting a run by strategy or by symbol
To start Optimize mode, click Optimize in the toolbar or choose Run / Optimize from the menu. The Optimize dialog appears before the test starts, giving you a chance to configure the run:
.png)
The sections below describe each area of the dialog. Most of these settings also have script-level equivalents — see OptimizeSettings for the full list.
Parameter List
The left panel lists every entry in the script's Parameters section, along with the number of values defined and a preview of the value list. The checkbox next to each parameter controls whether that parameter is varied during the run — unchecked parameters always use their default (first) value.
By default no boxes are checked initially. A parameter definition whose comment includes the special tag {X} is checked by default.
The Select All and Clear Selection buttons toggle every box at once.
Optimization Mode
This group chooses how the parameter space is searched:
•Combinatorial is the exhaustive nested loop — every combination of the checked parameter values is tested. Test counts grow quickly with the number of varied parameters, but RealTest can handle very large runs — portfolio-level optimizations involving thousands of tests across thousands of stocks over many years are quite practical. Whether this approach is likely to discover a strategy that will be profitable in the future is another matter.
•Sequential sweeps one parameter at a time. At the end of each sweep, the value that produced the highest score is locked in and the next parameter is swept. Two or three full iterations of a sequential run will often find a good neighborhood faster than the combinatorial approach.
•Genetic is a randomized search inspired by genetic optimization, but not a true genetic optimizer. Before each test, a random subset of parameters is set to new random values from their lists. (How many parameters change each step is controlled by the mutate% setting.) If the score improves, the change is kept; otherwise the prior best values are restored. Genetic mode typically converges on the best combinatorial result within roughly the square root of the total combination count. Requires Test Iterations.
•Random is similar to Genetic but every parameter is rerolled before every test and the score is ignored. A useful application is to set reasonable value ranges for each parameter, run 100 iterations, and look at the median result — this can give a reasonable estimate of how the system might perform in the future given how arbitrary parameter selection often is. Requires Test Iterations.
•Defaults Only is equivalent to unchecking every parameter — the script runs once using each parameter's default value. Combined with For Each Strategy, For Each Symbol, or Date Intervals, this is how to run a multi-test job with no parameter variation.
For Each Strategy / For Each Symbol
These checkboxes split the job into one test (or one whole optimization) per strategy or per symbol, as if every other strategy were commented out or every other symbol were absent from the data. They can be combined — doing so produces strategy × symbol tests.
Shard Processes
Specifying a count > 1 enables parallel sharded optimization — see Parallel Optimization (Shards) below.
Test Iterations
The number of times to run the configured test. Use this to:
•repeat a test that uses the Random function in one or more formulas, to obtain a range of potential outcomes
•specify how many tests Genetic mode should run
•specify how many tests Random mode should run
Score
The Results column to use as the optimization's fitness function. Because Results columns are formula-based, you can score by any expression — net profit, Sharpe, MAR, or any custom formula you have defined. The Score column is used to rank rows when Sort After Each Test is checked, to drive the Sequential and Genetic searches, and to select best parameter sets when building a Walk-Forward test.
Results Window
Controls how the Results window and the underlying RTR file are populated during the run.
•Clear Results Window — empty the window before the run starts (otherwise new test rows are appended to the existing list).
•Sort After Each Test — re-sort the rows by Score after each test completes. Pair this with Keep Best to retain only the top rows as the run progresses.
•Keep Best — keep only the N highest-scoring rows during the run. By Date applies the limit to each Date Interval separately, and By Symbol applies it to each For Each Symbol test. Requires Sort After Each Test.
•Keep No-Trade Tests — by default a parameter combination that produces zero trades is discarded. Check this to keep those rows.
•Discard Stat Details — drop the per-period statistics array from each saved test, keeping only the precomputed Results column values. This dramatically reduces RTR size and memory use, making optimizations of millions of tests feasible. Rows saved this way can no longer drive the Graph, Plot, Report, Save Stats, or Save Trades menu items — only their Results column values are available.
•Continue From File — pick up where a previously interrupted optimization left off. RealTest seeds the run from the parameter combinations already in the ResultsFile RTR and skips those combinations during enumeration. Requires Combinatorial mode, a ResultsFile setting, Test Iterations = 1, no Keep Best pruning, no Date Intervals, no For Each Strategy/Symbol, and no Walk-Forward. Optimizations that have a ResultsFile setting now save their RTR file frequently during the run, so a crash, power loss, or manual stop loses at most a few minutes of completed tests.
•Show — chooses when the Results window is opened. Live updates the window as each test finishes (the historical behavior). AtEnd runs the optimization headless and opens the RTR file when the run completes. None runs headless and never opens the file — useful for batch pipelines where the RTR is the final artifact. AtEnd and None require ResultsFile and skip the per-test list update, which materially speeds up very large optimizations.
Date Intervals
The Date Intervals panel is used primarily to generate a walk-forward test, but can also simply produce a series of results for different time periods. (It is OK to run a script in Optimize mode even if it contains no optimization parameters.) Setting the time unit to "Years" is a convenient way to produce one result row per calendar year of a date range.
By default each date interval is a sliding window of the same length. The Anchor setting optionally locks the start or end date. Anchoring the start date makes each interval longer than the preceding one; anchoring the end date does the opposite.
Create Walk-Forward
Generates a walk-forward test from the configured Date Intervals and parameter selection. See Walk-Forward Tests for a detailed description of this feature.
Output
•Multi Save As CSV — perform SaveStatsAs and/or SaveTradesAs once per test, creating separate files for each. These settings are otherwise ignored in a multi-test run.
•Summary Log — write a log window summarizing the optimization when the run finishes.
Maximum Test Count
The number in the lower-right corner of the dialog shows how many tests the current settings will produce. Watch it change as you toggle checkboxes and adjust options — it is the quickest way to see whether the run will take seconds, minutes, or days before committing to it.
Parallel Optimization (Shards)
For very large Combinatorial runs, the Shard Processes setting divides the parameter space across multiple parallel RealTest processes (2 to 32). Each shard runs an independent slice of the largest active parameter's value list and writes its own RTR file. When all shards complete, RealTest merges them into a single combined RTR.
Sharding has the following requirements:
•Combinatorial, Genetic or Random mode (sharding is not supported for Sequential mode).
•No Date Intervals, For Each Strategy, For Each Symbol, Walk-Forward.
•More total tests than shards.
If the dialog settings end up with nothing meaningful to slice across, the optimization runs inline in a single process instead, so OptShards is safe to leave set in scripts that may or may not need it.
In Combinatorial mode sharding can be combined with Continue From File to resume a previously stopped sharded run, provided the same shard count is used.
Running From a Script
Most of the choices in this dialog also have script-level equivalents in the OptimizeSettings section, all named with the Opt prefix. Settings made in the dialog become the new defaults for the next time the dialog opens, and settings made in the script become the defaults shown in the dialog when it opens.
Setting OptNoDialog: True in the script suppresses the dialog entirely — useful for unattended or batch-mode runs where every setting is specified in the script.
See also Tutorial 2 - Optimization for a specific usage example.
|
