Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify ux optimize performance of backtesting and livetrading #1148

Draft
wants to merge 66 commits into
base: master
Choose a base branch
from

Conversation

sgflt
Copy link
Contributor

@sgflt sgflt commented Apr 14, 2024

Fixes #1141
Fixes #1134.

Changes proposed in this pull request:

  • optimize performance of backtesting
    • Indicator.getStableBars() replaced by boolean isStable()
      • implemented propagation of stability
    • optimized running sums, uses previous partial sum
    • number factories caches constants with runtime defined precision
    • Optimized RecursiveIndicator -> no more StackOverflowException
    • Optimized LowestValueIndicator
  • optimize memory footprint of live trading
    • raspberry pi power save mode enabled
  • enhance parallelization to indicator level
  • bump to JDK 21
  • general code clean up
    • numeric indicators uses NumericIndicator as a base, not as wrapper
    • moving averages moved to own package
    • fixed javadocs
      • License headers should be comments
      • DifferenceIndicator
  • Used java.time.Instant as time source instead of ZonedDateTime

IDEAS:

  • untangle indicators from BarSeries but preserve Statefulness
    • allows creation of MACD from sun spot count and other sorcery things

Cons:

  • dropped serialization support

  • dropped mutable last bar (why trade on "unstable" candle?)

    • mutability has performance impact (maybe exists another solution)
  • I like the final API

  • added tests for stability API

  • all tests passed

  • Performance requirements met

  • added an entry with related ticket number(s) to the unreleased section of CHANGES.md

sgflt added 26 commits March 27, 2024 18:56
- also updated ancient dependencies
- longer barSeries may benefit from reusage of precomputed partial sums

- random access fallbacks to slowPath
- longer barSeries may benefit from reusage of precomputed partial sums

- random access fallbacks to slowPath
- cleaned tests
- use builders
- eliminated different parameter orders
- enforced usage of builder that belongs to series

- TODO remove constructors in BaseBar and use BaseBarBuilder
- assumed always filled hi, lo
- implementation does cont with nulls and  os builder does too
- no need to manually calculate endTime
…formance-bang

# Conflicts:
#	ta4j-core/src/main/java/org/ta4j/core/indicators/SMAIndicator.java
…breaking-performance-bang

# Conflicts:
#	ta4j-core/src/main/java/org/ta4j/core/indicators/SMAIndicator.java
#	ta4j-core/src/main/java/org/ta4j/core/indicators/helpers/RunningTotalIndicator.java
#	ta4j-core/src/test/java/org/ta4j/core/indicators/SMAIndicatorTest.java
- they work only in backtestig use case
- TODO shared cache
- TODO parallelized independent indicator calculation
@sgflt sgflt force-pushed the simplify-ux-optimize-performance-of-backtesting-and-livetrading branch from 9db6eae to d5367f9 Compare April 14, 2024 15:57
@TheCookieLab
Copy link
Member

This is too big of a change set. Please break it up into discrete chunks that can stand alone as value add.

@sgflt
Copy link
Contributor Author

sgflt commented May 14, 2024

Can't break PoC now, until results are verified.

@sgflt sgflt force-pushed the simplify-ux-optimize-performance-of-backtesting-and-livetrading branch from d203934 to 2ff9899 Compare May 14, 2024 10:45
- FIXED calculation that is comparable to VAR() in LibreOffice now
@sgflt sgflt force-pushed the simplify-ux-optimize-performance-of-backtesting-and-livetrading branch from 59b61fd to f916824 Compare May 14, 2024 18:00

Num variance = getBarSeries().numFactory().zero();
// cannot use RunningTotalIndicator because mean is changing each tick
final Num average = this.mean.getValue();
Copy link
Contributor Author

@sgflt sgflt May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look at https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance

Welford's online algorithm. Probably may be used to optimize redundant iterations over array.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve stability propagation Suggestion for improving the organization of the proyect folders and classes
3 participants