Model performance

[Haczan]

I have a problem with the poor performance of the trained model on new data.
On old data when checking the trained model the performance is good.

How can I improve model performance on new data?

I know that I can load ckpt and train the model with new data, but the model will only have good performance with past data and will have poor performance with future data.

Code I use to check the model:

import torch
from easydict import EasyDict
from ding.config import compile_config
from ding.policy import DQNPolicy, single_env_forward_wrapper
from ding.model import DQN
from dizoo.gym_anytrading.config.stocks_dqn_config import main_config, create_config
from dizoo.gym_anytrading.envs.stocks_env import StocksEnv


def main(main_config: EasyDict, create_config: EasyDict, ckpt_path: str):
    main_config.exp_name = 'stocks_dqn_deploy'
    cfg = compile_config(main_config, create_cfg=create_config, auto=True)
    env = StocksEnv(EasyDict({"env_id": 'stocks-v0', "eps_length": 253,\
            "window_size": 20, "train_range": None, "test_range": None, "stocks_data_filename": 'STOCKS_GOOGL_test'}))
    model = DQN(**cfg.policy.model)
    state_dict = torch.load(ckpt_path, map_location='cpu')
    model.load_state_dict(state_dict['model'])
    policy = DQNPolicy(cfg.policy, model=model).eval_mode
    forward_fn = single_env_forward_wrapper(policy.forward)

    obs = env.reset()
    returns = 0.
    while True:
        action = forward_fn(obs)
        obs, rew, done, info = env.step(action)
        returns += rew
        if done:
            break
    env.render()
    print(f'Deploy is finished, final epsiode return is: {returns}')


if __name__ == "__main__":
    main(main_config, create_config, "stocks_dqn_seed0/ckpt/ckpt_best.pth.tar")

Old data:

New data:

[Cloud-Pku]

This phenomenon is normal, because in a huge market with a long time span, the signal-to-noise ratio of the original features (close, open, volume...) of one stock is quite low. If you want to use the past data to train the model and make good profits on the future data, here are some suggestions:

1. Use factors with high signal-to-noise ratio as features, such as macd, rsi, cci...
2. Use appropriate normalization methods, such as overnight change rate of closing price.
3. Feature Neutralization.
4. When the training time is too long, the neural network will over fit the train data. From your picture, I think that the neural network at this time is fitting to noise.