FinRL is a quick way to try deep reinforcement learning on market data: it comes with trading environments, data downloaders and agents ready to train. Installing it is the hard part. It pulls in a large set of dependencies, many with upper version limits, it changes often, and installing it into an environment you already use can quietly downgrade NumPy and pandas under your other projects. This post is how I install it on a Mac: in a conda environment of its own, pinned to one commit, with the reason for each step, a check at the end and a first small agent to show that the pieces fit together. I ran the install steps in a new, empty conda environment on macOS 27.2.1 (Apple silicon) with Python 3.11.18, and the outputs below are the ones I got.
About FinRL
FinRL is an open-source library from the AI4Finance Foundation for deep reinforcement learning (DRL) in finance. It is organised in three layers: market environments (Gym-style simulators for tasks such as stock trading and portfolio allocation), DRL agents, and financial applications that join the two in a train, test and trade pipeline. Most of its agents come from three DRL libraries: Stable-Baselines3 and ElegantRL, both built on PyTorch, and Ray RLlib; a few, such as the portfolio-optimisation policies, are written in PyTorch inside FinRL itself. The tutorial in its README downloads Dow 30 prices from Yahoo Finance, adds technical indicators and trains A2C, DDPG, PPO, TD3 and SAC agents with Stable-Baselines3. See also the documentation.
As of November 2026 the README also points to FinRL-X (FinRL-Trading), a separate, production-oriented successor. The FinRL repository keeps the original framework for education and research, and it is the one this post installs. (I read the README on 15 November 2026.)
Installation on macOS (Apple silicon)
This post covers macOS on Apple silicon. FinRL's own packaging asks for Python 3.7 or newer (python_requires=">=3.7" in its setup.py, with classifiers up to 3.12). I use Python 3.11; when I ran the steps, asking conda for python=3.11 gave Python 3.11.18. Windows and Linux notes may follow; until then, FinRL's installation guide has sections for Ubuntu and Windows 10.
I ran Steps 4 to 7 in order, as written, in a new conda environment on macOS 27.2.1 (Apple silicon), with Python 3.11.18. The pinned commit, 7e98df4 (FinRL 0.3.8), was still the head of FinRL's master branch when I ran the steps. Steps 1 to 3 were already done on my Mac, so for those I ran only the checks. The install worked as written. The one gap is that FinRL does not install Plotly, which the chart at the end of this page needs (Step 7 adds it). Before that I had run the same steps as an upgrade of an older FinRL environment, which caused two other problems; they are in Troubleshooting as well.
Step 1: Install Anaconda
Make sure Anaconda is installed. To check, run this in a terminal:
conda --versionIt printed conda 26.5.3 for me. If it is not installed, download it from Anaconda's download page. Conda manages the Python version as well as the packages, which is what lets FinRL have an environment of its own in Step 4.
Step 2: Install Homebrew
Homebrew is the package manager I use for the system tools in Step 3. Install it with the one-line command on brew.sh. That command downloads Homebrew's install script from its GitHub repository and runs it with bash. The script explains what it will do and pauses before doing it, asks for your password so that it can create its folders (under /opt/homebrew on Apple silicon), and installs Apple's Command Line Tools for Xcode, which include the C and C++ compilers, if they are missing. Piping a download straight into a shell runs whatever the server sends, so read the script first if you prefer; the installation notes describe the alternatives. At the end the script prints the commands that add brew to your PATH. Run them, then check:
brew --versionHomebrew was already on my Mac, so I only ran the check, which printed Homebrew 7.0.12.
Step 3: Install system dependencies (CMake and OpenMPI)
brew install cmake openmpiCMake is a build tool that some packages with C or C++ code use when they have to be compiled from source because there is no ready-made wheel for your Mac. OpenMPI is an implementation of MPI, the message-passing library that mpi4py, and RL code built on it such as OpenAI Baselines, needs at build time; that is why FinRL's installation guide puts this command under "Install OpenAI". This step installs the two system tools, not Baselines itself. Both were already installed through Homebrew on my Mac, so I did not re-run the command. The clean install of the pinned commit did not use either of them: pip built only three packages itself, FinRL and ElegantRL (both pure Python, fetched from Git) and msgpack 1.0.3 (from its source archive), and it installed neither mpi4py nor Baselines. (It also reused a peewee wheel that it had built on this Mac before; with an empty pip cache it builds that one too.) You may not need this step, but I keep it because FinRL's guide still asks for it.
Step 4: Create a conda environment with Python 3.11
conda create -n finrl python=3.11
conda activate finrlFinRL gets an environment of its own because several of its dependencies have upper version limits that clash with an up-to-date setup. Python 3.11 is inside FinRL's stated range, and naming it in the command means the environment does not change when conda's default Python moves on. Run the remaining steps with this environment active. I named my environment finrl, as in the commands above, and ran each later command from a script through conda run -n finrl, which has the same effect as activating it; in a script, conda create also needs -y to skip its confirmation prompt.
Step 5: Install SWIG
conda install swigSWIG generates Python bindings for C and C++ libraries. FinRL's installation guide needs it to build Box2D (box2d-py), the physics engine behind some Gymnasium environments, and FinRL's requirements.txt lists it. In the new environment conda installed SWIG 4.5.1, with the pcre2 library it needs, in a few seconds. It also printed a WARNING that is only a tip about its conda-pypi beta, not a problem. The clean install of the pinned commit did not install Box2D, so this step matters only if you install Box2D-based environments later.
Step 6: Install FinRL, pinned to a commit
pip install "git+https://github.com/AI4Finance-Foundation/FinRL.git@7e98df461cbc7772f16a545a2f13b521e5d0b8f4"Without the @<commit> part, pip installs whatever FinRL's default branch holds on the day you run it, so the instructions could break without anything in this post changing. The commit above was the head of master when I ran the steps (FinRL 0.3.8). To try a newer FinRL, replace the hash with the one this command prints:
git ls-remote https://github.com/AI4Finance-Foundation/FinRL.git HEADIn the new environment the install took a little over a minute and finished with no errors, and pip check reported No broken requirements found. It installed 173 packages, among them Stable-Baselines3 2.9.2, PyTorch 2.15.0, Gymnasium 1.4.0, NumPy 2.4.6, pandas 2.3.3, yfinance 0.2.58 and SQLAlchemy 1.4.54. Along the way pip printed INFO: This is taking longer than usual, which only means that it was trying several combinations of versions; it finished on its own. That minute is with every file already in pip's download cache from my earlier tries, and the finished environment takes 2.1 GB, so a first install spends longer downloading.
The pin does not cover everything: FinRL asks for ElegantRL straight from ElegantRL's default branch, with no commit (elegantrl @ git+https://github.com/AI4Finance-Foundation/ElegantRL.git), so that one dependency can still change. Its head was commit 2422830 when I ran the steps, and that is what the new environment got, as ElegantRL 0.3.10. Pip fetches it only when ElegantRL is missing, so an environment that already has ElegantRL keeps whatever commit it has. Once the install works, record exactly what you got, so that you can rebuild the same environment later:
pip freeze > finrl-freeze.txtMy pip freeze after the install (195 packages)
This is the environment every output on this page came from: the install above, plus ipykernel and plotly from Step 7 and nbconvert, which I added only to run this page's notebook from the command line.
absl-py==2.5.1
aiodns==4.0.4
aiohappyeyeballs==2.7.1
aiohttp==3.14.5
aiohttp-cors==0.8.1
aiosignal==1.4.0
ale-py==0.12.1
alpaca-py==0.37.0
alpaca-trade-api==3.2.0
annotated-types==0.8.0
appnope==1.0.0
asttokens==3.0.2
attrs==26.1.0
beautifulsoup4==4.15.0
bleach==6.4.0
Bottleneck==1.6.1
ccxt==3.1.60
certifi==2026.10.23
cffi==2.1.1
charset-normalizer==3.5.2
clarabel==0.11.1
click==8.5.1
cloudpickle==3.1.2
colorful==0.5.8
comm==0.2.3
contourpy==1.3.3
cryptography==50.0.2
curl_cffi==0.17.0
cvxpy==1.9.3
cycler==0.12.1
debugpy==1.8.22
defusedxml==0.7.1
deprecation==2.1.0
distlib==0.4.4
ElegantRL @ git+https://github.com/AI4Finance-Foundation/ElegantRL.git@24228304867bdc80165de435a598ef90b1893598
empyrical-reloaded==0.7.2
exchange_calendars==4.13.3
executing==2.3.0
Farama-Notifications==0.0.6
fastjsonschema==2.22.3
filelock==4.2.10
finrl @ git+https://github.com/AI4Finance-Foundation/FinRL.git@7e98df461cbc7772f16a545a2f13b521e5d0b8f4
fonttools==4.68.0
frozendict==2.5.1
frozenlist==1.8.0
fsspec==2026.9.0
google-api-core==2.44.0
google-auth==2.62.2
googleapis-common-protos==1.76.0
grpcio==1.85.1
gymnasium==1.4.0
h11==0.16.0
highspy==1.16.0
idna==3.20
ijson==3.6.0
ipykernel==7.4.0
ipython==9.18.1
ipython_pygments_lexers==1.1.1
jedi==0.20.1
Jinja2==3.1.6
joblib==1.6.0
jqdatasdk==1.9.8
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
jupyter_client==8.10.0
jupyter_core==5.9.1
jupyterlab_pygments==0.3.0
kiwisolver==1.5.1
korean_lunar_calendar==0.4.0
Markdown==3.11.3
markdown-it-py==4.3.0
MarkupSafe==3.0.4
matplotlib==3.11.2
matplotlib-inline==0.2.2
mdurl==0.1.2
mistune==3.4.0
mpmath==1.3.0
msgpack==1.0.3
multidict==7.2.1
multitasking==0.0.13
narwhals==2.28.0
nbclient==0.11.0
nbconvert==7.17.2
nbformat==5.11.1
nest-asyncio2==1.7.4
networkx==3.6.1
niltype==1.0.2
numpy==2.4.6
opencensus==0.11.4
opencensus-context==0.1.3
opencv-python==5.0.0.93
opentelemetry-api==1.45.2
opentelemetry-exporter-prometheus==0.66b2
opentelemetry-proto==1.45.2
opentelemetry-sdk==1.45.2
opentelemetry-semantic-conventions==0.66b2
osqp==1.1.3
outcome==1.3.0.post0
packaging==26.3
pandas==2.3.3
pandas_market_calendars==5.6.1
pandocfilters==1.5.1
parso==0.8.7
peewee==3.17.3
pexpect==4.9.0
pillow==13.0.0
platformdirs==4.14.5
plotly==7.1.0
ply==3.11
prometheus_client==0.26.0
prompt_toolkit==3.0.53
propcache==0.5.4
proto-plus==1.29.0
protobuf==7.36.2
psutil==7.2.2
psycopg2-binary==2.9.13
ptyprocess==0.7.0
pure_eval==0.2.4
py-spy==0.4.3
pyarrow==26.0.1
pyasn1==0.6.5
pyasn1_modules==0.4.2
pycares==5.2.1
pycparser==3.12
pydantic==2.14.3
pydantic_core==2.50.4
pyfolio-reloaded==0.9.9
pygame-ce==2.5.8
Pygments==2.21.0
pyluach==2.3.0
PyMySQL==1.2.4
pyparsing==3.3.3
pyportfolioopt==1.6.0
PySocks==1.7.1
python-dateutil==4.0.0
python-discovery==1.9.5
python-dotenv==1.2.4
pytz==2026.6
PyYAML==6.0.1
pyzmq==27.3.0
qdldl==0.1.9.post1
ray==2.60.0
referencing==0.37.0
requests==2.34.2
rich==15.0.0
rpds-py==2026.9.1
sb3_contrib==2.9.2
scikit-base==0.13.2
scikit-learn==1.9.1
scipy==1.17.1
scs==3.3.2
seaborn==0.13.2
selenium==4.32.0
six==1.17.0
smart_open==9.0.4
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==4.1.1
sparsediffpy==0.6.1
SQLAlchemy==1.4.54
sseclient-py==1.9.0
stable_baselines3==2.9.2
stack-data==0.6.3
stockstats==0.5.4
sympy==1.14.0
tensorboard==2.21.0
tensorboard-data-server==0.7.2
tensorboardX==2.6.5
th==0.4.1
threadpoolctl==3.8.0
thriftpy2==0.7.1
tinycss2==1.5.1
toolz==1.2.0
torch==2.15.0
tornado==6.6.1
tqdm==4.70.1
traitlets==5.16.1
trio==0.35.0
trio-websocket==0.12.2
typing-inspection==0.4.5
typing_extensions==4.16.0
tzdata==2026.6
urllib3==1.26.20
virtualenv==21.15.6
wcwidth==0.9.4
webdriver-manager==4.1.2
webencodings==0.6.1
websocket-client==1.9.2
websockets==10.4
Werkzeug==3.1.9
wrapt==2.5.2
wrds==3.1.6
wsproto==1.3.2
yarl==1.25.1
yfinance==0.2.58Step 7: Check that it works
First, confirm which version pip installed. FinRL 0.3.8 has no finrl.__version__ attribute (its __init__.py does not define one), so ask pip instead:
$ pip show finrl
Name: finrl
Version: 0.3.8
Summary: FinRL: Financial Reinforcement Learning Framework. Version 0.3.5 notes: stable version, code refactoring, more tutorials, clear documentationThe Version line is the one to read; the summary still carries the notes from version 0.3.5. The rest of this page is a notebook, run in the same environment. FinRL installs neither a Jupyter kernel nor Plotly, which draws the chart at the end, so to run it as a notebook, add both:
pip install ipykernel plotlyThe smoke test, run as a script, needs neither. The cell below records what the notebook ran on:
import platform
from importlib.metadata import version
print("macOS", platform.mac_ver()[0], platform.machine())
print("Python", platform.python_version())
for package in ["finrl", "stable_baselines3", "gymnasium", "torch", "numpy", "pandas", "yfinance"]:
print(f"{package:18}{version(package)}")
macOS 27.2.1 arm64 Python 3.11.18 finrl 0.3.8 stable_baselines3 2.9.2 gymnasium 1.4.0 torch 2.15.0 numpy 2.4.6 pandas 2.3.3 yfinance 0.2.58
Then a smoke test. The next three cells download six months of daily prices for two stocks from Yahoo Finance (so they need an internet connection), add FinRL's default technical indicators and create the stock-trading environment, following the same steps as FinRL's own example scripts at the pinned commit. Pasted together into one file, they also run as a script, python finrl_smoke_test.py.
from finrl.config import INDICATORS
from finrl.meta.env_stock_trading.env_stocktrading import StockTradingEnv
from finrl.meta.preprocessor.preprocessors import FeatureEngineer, data_split
from finrl.meta.preprocessor.yahoodownloader import YahooDownloader
TICKERS = ["AAPL", "MSFT"]
START, END = "2024-01-01", "2024-07-01"
# 1. Download daily prices: one row per ticker per trading day.
prices = YahooDownloader(start_date=START, end_date=END, ticker_list=TICKERS).fetch_data()
print(prices.shape)
prices.head()
YF deprecation warning: set proxy via new config function: yf.set_config(proxy=proxy)
[*********************100%***********************] 1 of 1 completed
[*********************100%***********************] 1 of 1 completed
Shape of DataFrame: (248, 8) (248, 8)
| Price | date | close | high | low | open | volume | tic | day |
|---|---|---|---|---|---|---|---|---|
| 0 | 2024-01-02 | 183.264755 | 186.028933 | 181.537146 | 184.755430 | 82488700 | AAPL | 1 |
| 1 | 2024-01-02 | 363.117889 | 368.042749 | 359.103584 | 366.045381 | 25258600 | MSFT | 1 |
| 2 | 2024-01-03 | 181.892563 | 183.501712 | 181.083047 | 181.862948 | 58414500 | AAPL | 2 |
| 3 | 2024-01-03 | 362.853546 | 365.457949 | 360.807236 | 361.296785 | 23083500 | MSFT | 2 |
| 4 | 2024-01-04 | 179.582489 | 180.747384 | 178.565669 | 179.819409 | 71983600 | AAPL | 3 |
The two progress bars and the YF deprecation warning come from yfinance, not from anything wrong with the install: FinRL's downloader passes yfinance a proxy argument (empty here), which recent yfinance versions want set another way. The table has 248 rows, one per ticker for each of the 124 trading days in the first half of 2024, and FinRL's eight columns: the date, open, high, low and close prices (adjusted for dividends and splits), volume, ticker and day of the week (0 for Monday).
# 2. Add FinRL's default technical indicators (MACD, Bollinger bands, RSI, ...).
engineer = FeatureEngineer(use_technical_indicator=True, tech_indicator_list=INDICATORS)
features = engineer.preprocess_data(prices)
print(len(INDICATORS), "indicators:", INDICATORS)
Successfully added technical indicators 8 indicators: ['macd', 'boll_ub', 'boll_lb', 'rsi_30', 'cci_30', 'dx_30', 'close_30_sma', 'close_60_sma']
# 3. Keep the rows with START <= date < END (END itself is left out), sort them by
# date and then ticker, and number the trading days 0, 1, 2, ... so that every
# ticker on the same day shares one index. StockTradingEnv steps through that index.
data = data_split(features, START, END)
print(data.shape, "trading days:", data.index.nunique())
# 4. Create the trading environment and take its first observation.
n = len(TICKERS)
env = StockTradingEnv(
df=data,
stock_dim=n,
hmax=100,
initial_amount=1_000_000,
num_stock_shares=[0] * n,
buy_cost_pct=[0.001] * n,
sell_cost_pct=[0.001] * n,
reward_scaling=1e-4,
state_space=1 + 2 * n + len(INDICATORS) * n,
action_space=n,
tech_indicator_list=INDICATORS,
)
obs, info = env.reset(seed=0) # reset accepts a seed, but nothing in StockTradingEnv is random
print(len(obs), env.action_space)
(248, 16) trading days: 124 21 Box(-1.0, 1.0, (2,), float32)
That is the install working. FinRL has eight default indicators, so for two stocks the first observation has 1 + 2 × 2 + 8 × 2 = 21 numbers: the cash, the two closing prices, the two share counts and sixteen indicator values. The action space is a box of two values between −1 and 1, one per stock, which the environment scales by hmax into shares to sell (negative) or buy (positive).
A first FinRL agent
With the install checked, the next step is to train something. The cells below train a PPO agent (Schulman et al., 2017, here the Stable-Baselines3 implementation through FinRL's DRLAgent wrapper) on three stocks for 50,000 steps, then run it on the following year and compare it with buying and holding the same three stocks.
This shows that data, environment, training and backtest run end to end in the environment above. It is not investment advice and not evidence that the agent, or reinforcement learning, makes money. It is one seed, one short training run, three hand-picked stocks and one test year, with none of the checks a real evaluation would need.
Setting the agent's seed was not enough to make this repeatable. Two runs with the same seed, each downloading its own prices, ended the test year at $148,360 and $149,042, about 0.5% apart. Yahoo's adjusted prices differed between the two downloads, in about 2,300 of the 3,018 rows, by less than one part in a million, and training turned that into a different agent. With the prices saved once and read back, every run gave the same numbers. So the first cell downloads the prices only if there is no saved copy, and saves them to a local file. The file stays on your machine: it is left out of this site's repository and its built pages.
from pathlib import Path
import pandas as pd
from finrl.agents.stablebaselines3.models import DRLAgent
TICKERS = ["AAPL", "MSFT", "JPM"]
TRAIN_START, TRAIN_END = "2020-01-01", "2023-01-01"
TRADE_START, TRADE_END = "2023-01-01", "2024-01-01"
SEED = 0
TIMESTEPS = 50_000
# Download once and keep a local copy: the same request can come back with slightly
# different adjusted prices, and that alone changes the trained agent.
cache = Path("FinRLInstallation") / "_prices.csv"
if cache.exists():
prices = pd.read_csv(cache, float_precision="round_trip") # read back exactly what was saved
else:
prices = YahooDownloader(start_date=TRAIN_START, end_date=TRADE_END, ticker_list=TICKERS).fetch_data()
cache.parent.mkdir(exist_ok=True)
prices.to_csv(cache, index=False)
features = FeatureEngineer(use_technical_indicator=True, tech_indicator_list=INDICATORS).preprocess_data(prices)
train = data_split(features, TRAIN_START, TRAIN_END)
trade = data_split(features, TRADE_START, TRADE_END)
print("training days:", train.index.nunique(), " test days:", trade.index.nunique())
Successfully added technical indicators training days: 756 test days: 250
The environment is the one from the smoke test, with one change: $100,000 of starting cash instead of $1,000,000. The environment trades at most hmax = 100 shares of each stock a day, and 100 shares of each of these three cost about $34,000 at the start of 2020, so investing a million dollars would take the agent some six weeks of buying at the limit; $100,000 takes three days. print_verbosity=25 makes the environment print a summary every 25th training episode instead of every 10th, to keep the output short.
import time
n = len(TICKERS)
env_kwargs = dict(
stock_dim=n,
hmax=100,
initial_amount=100_000,
num_stock_shares=[0] * n,
buy_cost_pct=[0.001] * n,
sell_cost_pct=[0.001] * n,
reward_scaling=1e-4,
state_space=1 + 2 * n + len(INDICATORS) * n,
action_space=n,
tech_indicator_list=INDICATORS,
print_verbosity=25,
)
train_env, _ = StockTradingEnv(df=train, **env_kwargs).get_sb_env()
agent = DRLAgent(env=train_env)
model = agent.get_model("ppo", seed=SEED, verbose=0) # prints FinRL's default PPO settings
start = time.perf_counter()
model = DRLAgent.train_model(model, tb_log_name="ppo", total_timesteps=TIMESTEPS)
print(f"Trained for {TIMESTEPS:,} steps in {time.perf_counter() - start:.0f} s")
{'n_steps': 2048, 'ent_coef': 0.01, 'learning_rate': 0.00025, 'batch_size': 64}
day: 755, episode: 25 begin_total_asset: 100000.00 end_total_asset: 166518.19 total_reward: 66518.19 total_cost: 9856.31 total_trades: 1845 Sharpe: 0.659 =================================
day: 755, episode: 50 begin_total_asset: 100000.00 end_total_asset: 165157.65 total_reward: 65157.65 total_cost: 7443.19 total_trades: 1708 Sharpe: 0.650 =================================
Trained for 50,000 steps in 22 s
The dictionary is FinRL's default PPO settings, which get_model prints. Each block below it is the environment's summary of one training episode, a pass through the three training years: the account value at the start and end, the trading costs paid, the number of trades and a Sharpe ratio for that episode. Training took well under a minute. Stable-Baselines3 picks a CUDA GPU when there is one and the CPU otherwise, so on a Mac it trains on the CPU. Next, the trained agent trades the test year with no further learning, and the same three stocks bought on the first test day in equal amounts and held, with no trading costs, give a benchmark:
account, actions = DRLAgent.DRL_prediction(model, StockTradingEnv(df=trade, **env_kwargs))
closes = trade.pivot(index="date", columns="tic", values="close")
shares = (env_kwargs["initial_amount"] / n) / closes.iloc[0]
buy_and_hold = (closes * shares).sum(axis=1)
def summary(values):
values = pd.Series(values.to_numpy())
drawdown = values / values.cummax() - 1
return {
"final value ($)": round(values.iloc[-1]),
"total return (%)": round(100 * (values.iloc[-1] / values.iloc[0] - 1), 1),
"max drawdown (%)": round(100 * drawdown.min(), 1),
}
pd.DataFrame.from_dict(
{"PPO agent (seed 0)": summary(account["account_value"]), "Equal-weight buy and hold": summary(buy_and_hold)},
orient="index",
)
hit end!
| final value ($) | total return (%) | max drawdown (%) | |
|---|---|---|---|
| PPO agent (seed 0) | 148772 | 48.8 | -14.9 |
| Equal-weight buy and hold | 147596 | 47.6 | -10.7 |
Show code
import plotly.graph_objects as go
import plotly.io as pio
pio.renderers.default = "plotly_mimetype+notebook_connected"
fig = go.Figure()
fig.add_scatter(x=pd.to_datetime(account["date"]), y=account["account_value"], name="PPO agent (seed 0)")
fig.add_scatter(x=pd.to_datetime(buy_and_hold.index), y=buy_and_hold.to_numpy(), name="Equal-weight buy and hold")
fig.update_layout(xaxis_title="Date", yaxis_title="Account value ($)", hovermode="x unified")
fig.show()With seed 0 the agent ended 2023 at $148,772, up 48.8%, against $147,596 (up 47.6%) for buying and holding the three stocks in equal amounts. It was ahead for most of the year, but it also fell further in the autumn: its largest fall from a peak was 14.9%, against 10.7% for buy and hold.
Read nothing into this. 2023 was a strong year for these three stocks taken together, so almost anything that stayed invested made money. This is one seed of one short training run, tested on one year; the benchmark pays no trading costs while the agent pays 0.1% on every trade; and I picked three large companies that are still large today, which is a form of hindsight (survivorship bias). Telling whether an agent has learned anything useful needs several seeds, several test periods, realistic costs and a look at what it actually trades, which is more than an install post can do. What this does show is that the whole pipeline runs, in under a minute, in the environment above.
Troubleshooting
These are the problems I hit, and only those. The first came up in the clean install. The next two came up earlier, when I upgraded an existing environment that had FinRL 0.3.6; the fresh environment did not show them, but I keep them for anyone upgrading. The last is about repeating a run, and it happens on a clean install too.
The chart cell fails with No module named 'plotly'.
-
Symptom: in the new environment every cell ran until the chart, which stopped with
ModuleNotFoundError: No module named 'plotly'. -
Cause: FinRL does not depend on Plotly, so installing FinRL leaves it out. The older environment I used first already had Plotly, which hid this.
-
Fix: install it; this gave Plotly 7.1.0, and
pip checkstayed clean. Step 7 installs it together with the Jupyter kernel.pip install plotly
Every Yahoo download fails with a rate-limit error.
I hit this when upgrading an existing environment; a fresh environment did not show it. The clean install chose yfinance 0.2.58 and websockets 10.4 by itself, and all seven Yahoo downloads I made in it worked.
-
Symptom: the smoke test printed
['AAPL']: YFRateLimitError('Too Many Requests. Rate limited. Try after a while.'), the same forMSFT, and then FinRL stopped withValueError: no data is fetched.A single directyf.downloadcall a little later failed the same way. -
Cause: the environment still had yfinance 0.2.53 from the older install. FinRL 0.3.8 accepts any yfinance from 0.2 up to (not including) 0.3, so the upgrade left it alone. The error was not about how often I asked: the same request worked straight after upgrading yfinance.
-
Fix: upgrade yfinance within FinRL's range.
pip install --upgrade yfinancealone is not it: it would jump to yfinance 1.8.0, outside that range, and the newest 0.2 release (0.2.66) needswebsockets13 or newer, whilealpaca-trade-api(a FinRL dependency) needs a version below 11. Asking for both limits at once gave yfinance 0.2.58, which works, andpip checkstayed clean:pip install --upgrade "yfinance>=0.2,<0.3" "websockets<11"
pip list shows pyfolio and pyfolio-reloaded, empyrical and empyrical-reloaded, pygame and pygame-ce.
I hit this when upgrading, too; the fresh environment has only pyfolio-reloaded 0.9.9, empyrical-reloaded 0.7.2 and pygame-ce 2.5.8.
-
Symptom: after the upgrade, each pair was installed side by side, although
pip checkreported nothing. -
Cause: FinRL 0.3.6 used
pyfolio(which needsempyrical); 0.3.8 uses the maintained forkspyfolio-reloadedandempyrical-reloaded, and the extras of Stable-Baselines3 2.9.2, which FinRL installs, ask forpygame-ce, while the older environment hadpygame. Each fork installs into the same import name as the original (import pyfolio,import empyrical,import pygame), so pip wrote the new files over the old ones but left the old packages registered. Nothing needed the old three any more (pip showlisted nothing under "Required-by" forpyfolioandpygame, and onlypyfolioforempyrical), and uninstalling one of a pair later would delete files the other one uses. -
Fix: remove the old three, then reinstall the forks so that their files are complete again:
pip uninstall -y pyfolio empyrical pygame pip install --force-reinstall --no-deps pyfolio-reloaded==0.9.9 empyrical-reloaded==0.7.2 pygame-ce==2.5.8Afterwards
import pyfolio, empyrical, pygamegave versions 0.9.9, 0.7.2 and 2.5.8, andpip checkwas clean.
The same seed gives a different agent.
This one does happen on a clean install.
- Symptom: two runs of the agent above with
seed=0, each downloading its own prices, ended the test year at $148,360 and $149,042, about 0.5% apart, and their training summaries differed too. - Cause: the prices, not the seed. Two downloads of the same tickers and dates differed in about 2,300 of the 3,018 rows, by less than one part in a million, while the volumes were identical. That is rounding noise in Yahoo's adjusted prices, and training amplifies it. With the same saved prices, every run gave identical training summaries and results.
- Fix: download once, save the prices to a file and train from the file, as the first agent cell does.
Takeaways
- Give FinRL a conda environment of its own, and pin the install to a commit (
@7e98df4…, FinRL 0.3.8 here). The unpinned command installs whatever FinRL's default branch holds that day. ElegantRL still comes unpinned, so save apip freezeonce an install works. - On a fresh environment the pinned install worked as written, but FinRL installs neither Plotly nor a Jupyter kernel, so add them to run this page as a notebook. Upgrading an older FinRL environment is not the same as a clean install: pip keeps any old dependency that still fits the new limits, which left me a yfinance that Yahoo rejected and three pairs of packages sharing one import name. Run
pip check,pip listand a smoke test after every install or upgrade. - A seed alone does not make a FinRL run repeatable when the data is downloaded each time; save the prices once and train from the saved copy. Tested in a new conda environment with Python 3.11.18 on macOS 27.2.1 (Apple silicon); Windows and Linux are untested.
References
- AI4Finance Foundation, FinRL repository, README read on 15 November 2026 at commit
7e98df4, with thepyproject.tomlandsetup.pythat state its Python requirement and dependencies. - FinRL installation guide (macOS, Ubuntu and Windows 10) and documentation.
- X.-Y. Liu, H. Yang, Q. Chen, R. Zhang, L. Yang, B. Xiao and C. D. Wang, "FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance", Deep RL Workshop, NeurIPS 2020. arXiv:2011.09607.
- A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus and N. Dormann, "Stable-Baselines3: Reliable reinforcement learning implementations", Journal of Machine Learning Research 22(268), 2021. Documentation.
- ElegantRL and Ray RLlib, the other two agent libraries FinRL wraps.
- H. Yang, B. Zhang, Y. She, X. Liao and X. Zhang, "FinRL-X: An AI-native modular infrastructure for quantitative trading", 2026. arXiv:2603.21330; code in FinRL-Trading.
- Homebrew: brew.sh and its installation notes.
- pip documentation: VCS support (installing from a Git commit) and
pip install(including--dry-run). - J. Schulman, F. Wolski, P. Dhariwal, A. Radford and O. Klimov, "Proximal policy optimization algorithms", 2017. arXiv:1707.06347.
- yfinance, the Yahoo Finance downloader FinRL's
YahooDownloaderuses.