[BUG] VersusEnterAndHoldCriterion misleading for "+/-"-criteria

[nimo23]

Describe the bug
Some criteria values can have positive and negative values (e.g. PnlCriterion): For such criteria, the VersusEnterAndHoldCriterion compares X with its EnterAndHold, but it does not handle the sign (+/-) in favour of X.

1. Example: Given:
myPnl +4%
enterAndHoldPnl: - 4%
=> actual calculation = myPnl / enterAndHoldPnl = 4 / -4 = -1 (it should be +1, because -1 would mean X is not better than its EnterAndHold)

2. Example: Given:
myPnl -4%
enterAndHoldPnl: +4%
=> actual calculation = myPnl / enterAndHoldPnl = -4 / +4 = -1 (correct, because -1 means X is not better than its EnterAndHold)

3. Example: Given:
myPnl -8%
enterAndHoldPnl: -4%
=> actual calculation = myPnl / enterAndHoldPnl = -8 / -4 = 0.5 (ok?)

Expected behavior
If a criterion can have both positive and negative values, then VersusEnterAndHoldCriterion should handle it in a special way so that the result of the VersusEnterAndHoldCriterion can be interpreted unequivocally.

Maybe we could use another comparision method (instead of x.dividedBy(enterAndHold), we could use "euclidian distance" (but then we have no sign) or any other representative distance calculation, maybe from package criterion/statistics).

[phong-phuong]

Correct calculation is: (Strategy Account Balance)/(Buy and Hold Account Balance) - 100%.
In your first example, it would look like:
(1.0 + 0.04)/(1.0 - 0.04) - 1.0 = 1.04/0.96 - 1.0 = 1.08333333 - 1.0 = 0.08333333, i.e. The strategy when compared to Enter and Hold, outperforms it by 8.33%.

[nimo23]

@phong-phuong good idea. I am currently working on a PR to fix this bug.

The formula

(1 + x) / (1 + enterAndHold) - 1

can only be used if x and enterAndHold are percentage values, or?

For example, look at the NumberOfWinningPositionsCriterion which returns absolute numbers:

Given:

myNumberOfWinnings = 9
enterAndHoldWinnings = 3
=> vsEnterAndHold = (1 + 9)/ (1 + 3) - 1 = 1.5 (what does this mean?)
=> vsEnterAndHold = 9/3 = 3 (which means "x" is 3 times better than "enterAndHold", i.e. it outperforms it by 300%)

So we must differ if the criterion is a percentage value or not and then provide two distinct formulas:

• for percentage values, we use: (1 + x) / (1 + enterAndHold) - 1
• for non percentage values, we use: x / enterAndHold

Am I right?

[phong-phuong]

Not quite, the formula will depend on what is being compared, and whether "returns" come into the picture.

Consider a portfolio with a start capital of $100.
Strategy A ends with or returns $104 back to the user at the end of a backtest.
Strategy B ends with or returns $96 back to the user at the end of a backtest.

In this context of returns, the return for Strategy A is 104%, and return for Strategy B is 96% which is what ta4j uses.

In Absolute values:
Strategy A Profit: $4
Strategy B Profit: -$4

In percentages:
Strategy A Profit: 4% or 0.04
Strategy B Profit: -4% or -0.04

Absolute difference between Strategy A and Strategy B:
$4 - -$4 = $8
Strategy A outperforms strategy B by $8.

But what if we wanted to compare the profits generated by Strategy A compared to Strategy B?
Well, since we already know Strategy A outperformed by $8, and Strategy return $4, 8/4 = 2, so in this the profits made by Strategy A is 2x Strategy B, in other words, twice as much or 200% of Strategy B.

If we wanted to compare returns (the final account balance) from strategy A over strategy B as a ratio?
104/96 = 1.08333

If we wanted to compare returns (the final account balance) from strategy A over strategy B but exclude the initial cost of investment?
104/96 - 1= 0.08333

In the number of winnings example you gave:
Strategy A: 9 wins
Strategy B: 3 wins
Strategy A has 3x more wins than strategy B.

The formula: (1+9)/(1+3) -1 doesn't come into play, nor make sense because we are not talking about returns or Return on Investment.
We are only doing a direct comparison.

Hope that clears any confusion.

[nimo23]

The formula: (1+9)/(1+3) -1 doesn't come into play, nor make sense because we are not talking about returns or Return on Investment.

To abstract your last phrase: "..because we are not talking about percentage values (e.g. Returns, etc.) but absolute values (e.g. NumberOfWinningPositions, etc.)".

That's what I want to find out and what I asked for. The formula (1 + x) / (1 + enterAndHold) - 1 is only suitable for percentage values (e.g. returns, etc.) but not for absolute values.

Either we find a formula which can be used for both, percentage and absolute values, or the VersusEnterAndHoldCriterion must know beforehand if the Criterion to compare calculates percentage or absolute values, so that VersusEnterAndHoldCriterion is able to adapt its internal calculation.

For example:

public VersusEnterAndHoldCriterion(Criterion criterion, boolean isPercentageCriterion){
..
}

I think, that's the proper way, or?

[phong-phuong]

No, the formula you are referring to can be used with both percent and absolute values. The formula measures the difference in change compared to a base value.

Going back to the winnings example:
Strat A: 9 wins
Strat B: 3 wins

Applying the formula:
(Ending value - Starting value)/(Starting value)
= (3-9)/9 = -6/9 = -0.667

Or the other version of the formula:
(Ending value)/(Starting value) - 1
= (3/9) - 1 = 0.333 - 1 = -0.667

Which is interpreted as strategy B is 67% worse than the result obtained using strategy A.
Strategy A of 9 wins being the base comparison value or starting value.

[nimo23]

No, the formula you are referring to can be used with both percent and absolute values. The formula measures the difference in change compared to a base value.

I was referring this formula (which you introduced in #1073 (comment))

(1 + x) / (1 + enterAndHold) - 1

using this with the example:

myNumberOfWinnings = 9
enterAndHoldWinnings = 3
=> vsEnterAndHold = (1 + 9)/ (1 + 3) - 1 = 1.5 (what does this mean?)

I already explained that in #1073 (comment).

[phong-phuong]

Or going back to the winnings example where we compare the users strategy vs enter and hold:

User strategy: 9 wins
Enter and hold: 3 wins

Apply the formula:
(User strategy)/(Enter and hold) - 1
= 9/3 - 1 = 3 - 1 = 2

Which is interpreted as the user's strategy when compared to enter and hold is 2x more than the results of Enter and hold.

[phong-phuong]

The thing is, sometimes we want to include the base and sometimes we don't

[nimo23]

Apply the formula:
(User strategy)/(Enter and hold) - 1
= 9/3 - 1 = 3 - 1 = 2

I know, this is common sense:)

I only got irritated because you bring the formula with the base (= 1+)

(1 + x) / (1 + enterAndHold) - 1

sometimes we want to include the base and sometimes we don't

Yes, this is the primary question. When and why do I need to include the base.

[phong-phuong]

That's not the actual formula to be used every time, but I used that + 1 as a substitute of the initial balance because we don't know the starting balance of the user. It is only there to compare the ending capital of the two strategies when we only have the profit or percent to start with.

Compare account balances and directly comparing profits are two separate comparisons.

[nimo23]

Ok, I think there are misunderstandings due to a lack of information:

When we compare x versus EnterAndHold, then the EnterAndHold always represents 1 (= 100%), i.e. if x >1,then x performed better than its EnterAndHold and vice versa. To be able to do so, we need to include the base for EnterAndHold. The other thing why we need the base is to avoid minus signs canceling out when dividing.

[nimo23]

Example for +/--criteria:

x versus its EnterAndHold
=> ratio = x / EnterAndHold
=> 1. ratio is positive for -x / -EnterAndHold
=> 2. ratio is negativ for -x / +EnterAndHold
=> 3. ratio is negativ for +x / -EnterAndHold