Q22 Quick Start S&P500¶
Introduction to S&P500 stocks, a quick-start example strategy that dynamically selects low-volatility stocks and allocates capital based on their transaction volume.
You can clone and edit this example there (tab Examples).
Introduction to S&P500 stocks¶
This template demonstrates how to build a trading strategy, specifically designed for use with a dataset containing historical constituents of the S&P 500. The strategy can only trade stocks that are part of the S&P 500 index, and only when the stock is considered liquid. A company is deemed liquid if it is included in the S&P 500 at the observed point in time.
The strategy takes a low-risk approach by allocating weights to stocks based on their volatility. It assigns capital only when the ratio of the Average True Range (ATR(14)) to the closing price is below 0.0205, signaling lower volatility. The allocation is further refined by the stock's transaction money flow, where stocks with higher liquidity and trading volume receive a larger portion of the capital (weight). This approach ensures that more capital is directed towards assets with higher market activity, while also maintaining a risk-aware allocation.
Important Considerations:
- Trade only on historical S&P 500 stocks, and only when they are considered liquid (i.e., part of the index at the time of observation).
- The in-sample period begins on 2006-01-01. Earlier data may be used for testing and training purposes.
- The strategy must achieve a minimum in-sample Sharpe Ratio of 0.70 to be considered valid.
- The maximum allocation to any single asset is capped at 10% of total capital. If a weight exceeds this, it will be limited to 0.1.
- Manual stock selection or direct hand-picking is not permitted. The allocation process must be automatic.
- The strategy can open both long and short positions.
For official Q22 rules, click here.
Accessing the S&P 500 Stocks Dataset - Data loading¶
For the Q22 contest, a new dataset of S&P 500 stocks is available. You can obtain this dataset by calling the function stocks_load_spx_data() from the data module:
snp_stocks_data = qndata.stocks_load_spx_data(min_date='2005-06-01')
To check the list of S&P 500 companies starting from 2006-01-01, along with additional stock information, use the load_list() function:
snp_stocks_list = qndata.stocks_load_spx_list()
Quantiacs also provides data for other instruments, such as cryptocurrencies, indices, futures, as well as additional fundamental stock data and macroeconomic data. Note that there is also a Nasdaq-100 dataset available, and it is not a subset of the S&P 500. However, there is a significant overlap between the two.
The data is provided in xarray.DataArray format. Check here for more details on manipulating xarray data.
Technical analysis - indicators
Once the data is loaded, various technical indicators from the qnt.ta module can be used to generate trading signals. For a complete list of indicators and examples of their implementation, check the documentation page.
Trading strategy - algorithm¶
# Necessary imports
import xarray as xr
import numpy as np
import qnt.stats as qnstats
import qnt.data as qndata
import qnt.output as qnout
import qnt.ta as qnta
import qnt.backtester as qnbt
import qnt.graph as qngraph
def strategy(data, state=None, mfsp=135, limit=0.0205): # parameter state is used in the backtester (multi-pass) for stateful strategies
vol = data.sel(field="vol") ### Volume values
liq = data.sel(field="is_liquid") ### Liquidity values, 1.0 or 0.0 -> True or False
close = data.sel(field="close") ### close prices
high = data.sel(field="high") ### daily high
low = data.sel(field="low") ### daily low
atrs = qnta.atr(high=high, low=low, close=close, ma=14) ### Average True Range of 14 bars (working days)
ratio = atrs / close ### indicator
weights = xr.where(ratio > limit, 0, 1) ### Strategy condition - zero weight if ratio is bigger than limit
money_vol = vol * liq * close ### Daily money flow per liquid asset
total_money_vol = money_vol.sum(dim='asset', skipna=True) ### Total daily money flow of liquid assets
money_vol_share = money_vol / total_money_vol ### Daily money flow share per liquid asset
mvs_mov = qnta.sma(money_vol_share, mfsp) ### weights allocation by average money flow share in period of 135 days
return mvs_mov * weights
Weights - Single / Multi pass approach¶
In Single pass backtesting, which is significantly faster, weights are calculated using the entire dataset in a single run.
Multi pass backtesting evaluates weights on a day-by-day basis by slicing the dataset for each individual day.
If applicable, we recommend using the single pass approach for efficiency, while verifying strategy statistics with the multi pass backtester. If the statistics from single pass and multi pass match exactly, it indicates that forward-looking bias has likely not been introduced, and the strategy can be confidently submitted as single pass.
In rare cases, even if results are identical between single and multi pass, forward-looking bias might unintentionally occur (e.g., by incorporating global data variables into the logic). Such issues are generally mitigated in production but can result in discrepancies in statistics comparing development and production results.
### SINGLE PASS ###
## The min_date parameter is set to '2005-06-01' to ensure that the dataset includes at least 135 bars (approximately 190 days) of data,
## which is necessary for creating indicators and determining weights for the strategy.
snp_stocks_data = qndata.stocks_load_spx_data(min_date='2005-06-01')
weights = strategy(snp_stocks_data)
weights = qnout.clean(weights, snp_stocks_data) # fix liquidity
0% (0 of 367973) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (367973 of 367973) |################| Elapsed Time: 0:00:00 Time: 0:00:00
0% (0 of 79197) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (79197 of 79197) |##################| Elapsed Time: 0:00:00 Time: 0:00:00
0% (0 of 13033356) | | Elapsed Time: 0:00:00 ETA: --:--:--
39% (5213320 of 13033356) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
81% (10687306 of 13033356) |######### | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033356 of 13033356) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 1/13 1s
0% (0 of 13033360) | | Elapsed Time: 0:00:00 ETA: --:--:--
43% (5734652 of 13033360) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
86% (11338971 of 13033360) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033360 of 13033360) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 2/13 3s
0% (0 of 13033356) | | Elapsed Time: 0:00:00 ETA: --:--:--
39% (5213320 of 13033356) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
82% (10817639 of 13033356) |######### | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033356 of 13033356) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 3/13 4s
0% (0 of 13033360) | | Elapsed Time: 0:00:00 ETA: --:--:--
41% (5473986 of 13033360) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
86% (11338971 of 13033360) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033360 of 13033360) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 4/13 5s
0% (0 of 13033328) | | Elapsed Time: 0:00:00 ETA: --:--:--
43% (5734652 of 13033328) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
86% (11338971 of 13033328) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033328 of 13033328) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 5/13 7s
0% (0 of 13033248) | | Elapsed Time: 0:00:00 ETA: --:--:--
43% (5734608 of 13033248) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
86% (11338884 of 13033248) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033248 of 13033248) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 6/13 8s
0% (0 of 13033248) | | Elapsed Time: 0:00:00 ETA: --:--:--
44% (5864940 of 13033248) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
87% (11469216 of 13033248) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033248 of 13033248) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 7/13 9s
0% (0 of 12985944) | | Elapsed Time: 0:00:00 ETA: --:--:--
44% (5843655 of 12985944) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
88% (11557451 of 12985944) |########## | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (12985944 of 12985944) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 8/13 10s
0% (0 of 13033248) | | Elapsed Time: 0:00:00 ETA: --:--:--
44% (5864940 of 13033248) |##### | Elapsed Time: 0:00:00 ETA: 00:00:00
82% (10817556 of 13033248) |######### | Elapsed Time: 0:00:00 ETA: 0:00:00
100% (13033248 of 13033248) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 9/13 11s
0% (0 of 13033248) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (13033248 of 13033248) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 10/13 12s
0% (0 of 13033248) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (13033248 of 13033248) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 11/13 14s
0% (0 of 13033248) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (13033248 of 13033248) |############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 12/13 15s
0% (0 of 7637732) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (7637732 of 7637732) |##############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 13/13 16s
Data loaded 16s
Output cleaning...
fix uniq
ffill if the current price is None...
Check liquidity...
Ok.
Check missed dates...
Ok.
Normalization...
Output cleaning is complete.
When using the backtester for weight calculation, there is no need to manually load data or define a "load_data" function as input. It is only required to specify the competition_type (in this case, "stocks_s&p500"), and the corresponding data will be loaded automatically by default.
### MULTI PASS ###
w=qnbt.backtest(
competition_type="stocks_s&p500",
lookback_period=250,
start_date="2006-01-01",
strategy=strategy,
analyze=True,
)
Run last pass...
Load data...
0% (0 of 79197) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (79197 of 79197) |##################| Elapsed Time: 0:00:00 Time: 0:00:00
0% (0 of 5655064) | | Elapsed Time: 0:00:00 ETA: --:--:--
86% (4919850 of 5655064) |############ | Elapsed Time: 0:00:00 ETA: 00:00:00
100% (5655064 of 5655064) |##############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 1/1 25s
Data loaded 25s
Run strategy...
Can't load state. [Errno 2] No such file or directory: '/root/state.in.pickle.gz'
Load data for cleanup...
0% (0 of 1328396) | | Elapsed Time: 0:00:00 ETA: --:--:--
100% (1328396 of 1328396) |##############| Elapsed Time: 0:00:00 Time: 0:00:00
fetched chunk 1/1 7s
Data loaded 7s
Output cleaning...
fix uniq
ffill if the current price is None...
Check liquidity...
Ok.
Check missed dates...
Ok.
Normalization...
Output cleaning is complete.
Write result...
Write output: /root/fractions.nc.gz
Statistics¶
While the multi pass approach automatically displays the strategy’s statistics through the backtester, single pass mode only calculates the weights, so visualizing the stats must be done manually. You can use the helper print_stats function for this purpose:
def print_stats(stat):
display(stat.sel(time=slice('2006-01-01', None)).to_pandas().tail(10))
performance = stat.to_pandas()["equity"]
qngraph.make_plot_filled(performance.index, performance, name="PnL (Equity)", type="log")
def get_sharpe(stat):
return stat.isel(time=-1).sel(field="sharpe_ratio").item()
stat = qnstats.calc_stat(snp_stocks_data, weights.sel(time=slice('2006-01-01', None)))
print_stats(stat)
| field | equity | relative_return | volatility | underwater | max_drawdown | sharpe_ratio | mean_return | bias | instruments | avg_turnover | avg_holding_time |
|---|---|---|---|---|---|---|---|---|---|---|---|
| time | |||||||||||
| 2025-01-31 | 2.273613 | -0.001635 | 0.046905 | -0.010133 | -0.075852 | 0.939246 | 0.044056 | 1.0 | 494.0 | 0.028093 | 27.554153 |
| 2025-02-03 | 2.274708 | 0.000482 | 0.046901 | -0.009657 | -0.075852 | 0.939706 | 0.044073 | 1.0 | 494.0 | 0.028091 | 27.558328 |
| 2025-02-04 | 2.274269 | -0.000193 | 0.046896 | -0.009848 | -0.075852 | 0.939377 | 0.044053 | 1.0 | 494.0 | 0.028090 | 27.553333 |
| 2025-02-05 | 2.277483 | 0.001413 | 0.046892 | -0.008448 | -0.075852 | 0.940907 | 0.044121 | 1.0 | 494.0 | 0.028091 | 27.570238 |
| 2025-02-06 | 2.278164 | 0.000299 | 0.046887 | -0.008152 | -0.075852 | 0.941154 | 0.044128 | 1.0 | 494.0 | 0.028092 | 27.570363 |
| 2025-02-07 | 2.276589 | -0.000692 | 0.046883 | -0.008838 | -0.075852 | 0.940235 | 0.044081 | 1.0 | 494.0 | 0.028093 | 27.565531 |
| 2025-02-10 | 2.277380 | 0.000348 | 0.046878 | -0.008493 | -0.075852 | 0.940539 | 0.044090 | 1.0 | 494.0 | 0.028090 | 27.565838 |
| 2025-02-11 | 2.279543 | 0.000950 | 0.046873 | -0.007552 | -0.075852 | 0.941538 | 0.044133 | 1.0 | 494.0 | 0.028090 | 27.561900 |
| 2025-02-12 | 2.278310 | -0.000541 | 0.046869 | -0.008089 | -0.075852 | 0.940799 | 0.044094 | 1.0 | 494.0 | 0.028089 | 27.560579 |
| 2025-02-13 | 2.281425 | 0.001367 | 0.046864 | -0.006732 | -0.075852 | 0.942276 | 0.044159 | 1.0 | 494.0 | 0.028089 | 27.804556 |
get_sharpe(stat)
0.9422757452955967
Submit strategy to the competition¶
Use Submit button on my strategies page
Make sure that qnout.write(weights) has been added to cell, and the weights have been written. It is not required when using Multi pass backtester.
Note: After submitting the strategy to the contest, any weight exceeding 0.1 will be capped at that limit. You can apply weight normalization functions before submission, which may be more suitable, such as those that maintain the ratio between allocated weights. Details on these functions can be found here.
### Run this cell to get all weights below 0.1. This "hard cut" way is used on production if any weight exceeds 0.1.
### Any other normalization method can be used.
import qnt.exposure as qnexp
weights_capped = qnexp.cut_big_positions(weights=weights, max_weight=0.1)
Compare weights before and after normalization. Check the statistics again, it can change not only in negative way.
qnout.write(weights_capped)
Write output: /root/fractions.nc.gz
Common Reasons for Submission Rejection and Their Solutions¶
Here are some of the frequent reasons causing submission rejection in algorithmic trading competitions, and their corresponding remedies.
1) Missed call to write_output
Save algorithm weights, run code
qnt.output.write(weights)
2) Not eligible send to contest. In-Sample Sharpe must be larger than 0.7
Improve your algorithm. For example, you can use sections and get an algorithm that will pass the filter
- Example Trading System Optimization
- Example of a strategy using technical analysis indicators
Need help? Check the Documentation and find solutions/report problems in the Forum section.
3) Not enough bid information.
Run code
min_time = weights.time[abs(weights).fillna(0).sum('asset')> 0].min()
min_time
min_time must be less than or equal to January 1, 2006.
If min_time is larger than the starting date, we recommend to fill the starting values of the time series with non-vanishing values, for example a simple buy-and-hold strategy.
def get_enough_bid_for(data_, weights_):
time_traded = weights_.time[abs(weights_).fillna(0).sum('asset') > 0]
is_strategy_traded = len(time_traded)
if is_strategy_traded:
return xr.where(weights_.time < time_traded.min(), data_.sel(field="is_liquid"), weights_)
return weights_
weights_new = get_enough_bid_for(data, weights)
weights_new = weights_new.sel(time=slice("2006-01-01",None))