Refactor of MASVS / MASVS v2

[TheDauntless]

This post was written in collaboration with @sushi2k and @cpholguera

Current Problems

Context:

• @cpholguera and myself use the MASVS to validate mobile apps from an external perspective as consultants
• @sushi2k uses the MASVS from an internal perspective

Based on our experiences, the following issues often pop up:

• Chapter 1 (Architecture) contains multiple controls that cannot be validated from an external perspective. As a result, it's difficult to explain why we can't "cover the full MASVS" for an external assessment.
• It's not clear which controls can be automated / validated blackbox / need a full source code review
• There is overlap with the ASVS, which is much more thorough on various backend vulnerabilities
• Some controls are very broad and require very broad test-cases with multiple parts
• Some controls are overlapping (e.g. 'up to date security libraries (network-6)' and 'all 3rd party components are up to date(code-5)'

Proposal

We would like to refactor the MASVS with the following goals:

• Remove overlap with ASVS and focus on mobile-specific controls
• Make controls more specific so that they can be linked to very concrete and actionable MSTG test cases. This will coincide with a refactor of MSTG as well. The idea is that if a specific item needs to be tested in two very different ways, they should be split up.
• Remove redundancies/overlap
• Make sure there are clear divisions in the controls. For example, it should be clear which controls can be evaluated automatically, or which controls require a full source code review

Consequences

A refactor like this would be pretty invasive for the MASVS, but we do believe it is long overdue. It would make sense to go to an MASVS v2 (currently at v1.2), potentially also updating the identifiers to MASVS-XX (rather than MSTG-XX).
Together with this MASVS refactor, there will also be a refactor of the MSTG to make sure that every MASVS control has a clearly defined test-case.

Next steps

This is a pretty big undertaking and it would probably be chaos to do this refactor with many people. Because of this, we will create a first draft of MASVS v2 and present that for community feedback. In this first draft, we will try to tackle the issues listed above, as well as any issues that pop up as a result of this post.

Open questions

While we do have quite a good idea of the direction we want to go in, this is a community project so we want community input:

• Are there other grievances with the current version of the MASVS?
• Do you have an issue with a specific MASVS controls and why? (We did have a nice brainstorm on this already, but feel free to make suggestions)
• Are there issues that make it difficult to use the MASVS in a specific case?
• Do you have suggestions to pick up in the refactor?

[julepka]

Hi MASVS team!

Thank you for your hard work on improving MASVS! I really like your v2 idea and I wanted to share some thoughts.

Removing overlap with ASVS sounds reasonable because ASVS describes backend-related requirements in more detail. Also, ASVS has more information about architecture and threat modeling. My only concern here is that it will be possible for companies to request review/audit based on MASVS (mobile-only) without ASVS (backend). I think that it is important to underline that your mobile app can't have a good security level if you ignore ASVS.

Other thoughts about current MASVS:

• Translation quality. I know Russian as a native speaker and I see that some requirements are missing the main point of the original English requirements. Improve Russian Translation #567
• R-level requirements (reverse engineering and tampering). The R requirements are not that equal, in my opinion. I usually split them similarly to L1 and L2. For example, R-L1 is jailbreak/root detection, anti-debugger, reverse engineering tools detection. R-L2 is having more than 1 mechanism for each L1 requirement, obfuscation, etc.
• And some additional info on Hybrid apps as we discussed in Slack :)

[julepka]

One more note from me :)

MSTG-STORAGE-4: No sensitive data is shared with third parties unless it is a necessary part of the architecture.

I always had a feeling that this requirement doesn't belong to Data Storage section. It sounds to me like an ARCH requirement. Usually, to fulfill it I do a list of third parties the app is using and I list all the data shared with them. Sounds more like a general, documents-related, privacy-related, IP-related thing.

Also, from my standpoint, privacy-related requirements are almost a separate part of the evaluation:

• I check if Terms and Conditions and Privacy Policy are accepted before the user starts using the app.
• I create a document that describes how to fill in the Apple privacy form in the App Store.
• I check what data is shared with third parties, list it, identify any PII.
• I check if that if app requests PII then it should have a reason for it and I document it.
• I create a document to list all privacy regulations the app needs to comply with.
  As for me, mobile app privacy requirements are different from what we have for the web, and it may be useful to gather such requirements in a single place.

[TheDauntless]

Interesting input! This is indeed something we should look at. Arch seems like a good fit :)

[dzan]

Hi team!

Great initiative, in general especially these two concern I can relate to:

• It's not clear which controls can be automated / validated blackbox / need a full source code review
• Some controls are very broad and require very broad test-cases with multiple parts

Furthermore, zooming in on things I know best I fully agree with @julepka 's comment:

R-level requirements (reverse engineering and tampering). The R requirements are not that equal, in my opinion. I usually split them similarly to L1 and L2. For example, R-L1 is jailbreak/root detection, anti-debugger, reverse engineering tools detection. R-L2 is having more than 1 mechanism for each L1 requirement, obfuscation, etc.

I think it could be very interesting to conceptually split the requirements into 2 groups;

1. requirements on the various threats or risks to cover
2. requirements on the approach taken and the resilience of the implemented techniques

Covering the first group is done based on business 'needs', i.e. what is relevant for the kind of application in question and the actions it enables, data it processes,... If for example the sole concern is hiding client-side logic than maybe impeding comprehension is sufficient and ensuring application integrity is less relevant.

Covering the second group of requirements is driven more by the level of risk for the organisation, the expected complexity of adversary tools, time invested by reverse engineers,.. In some way the extent of covering the second group of requirements is more or less equal to the reasoning for L1 and L2 levels currently.

Making this more tangible with some (non-exhaustive and quickly put together) examples;

group 1:

Similar to the current buckets such as eavesdropping, comprehension, device binding,.. Typically I split the current 'dynamic analysis & tampering' in 3 further buckets;

• Code integrity, i.e. is the code being executed the one I built? Integrity of application code & data, indirections within the code, loaded libraries,...
• Environment integrity, i.e. is my application running in a platform 'stock' environment? Typical JB/root detection, debugger, emulation, integrity of system libraries,...
• Application integrity, i.e. is the application package the original one I produced? Package checksums, signatures, assets/resources,...

group 2:

Includes current 8.7 - 8.9 but could be greatly extended, e.g.;

• any runtime verification is inlined in existing and doesn't have its own signature
• every build of the application relocates runtime checks to different regions
• there are a set of different checks for each of the aspects in group 1
• every check has some form of polymorphism applied such that pattern matching becomes infeasible
• there is multitude of checks applied such that manual find & patch approaches are not feasible
• if a check leads to a crash these are again random and unpredictable
• if a check leads to a callback there is a data-flow rather than control-flow connection between the detection and the reaction
• techniques are in place to limit the in-memory lifetime of decrypted data/strings
• ...

Note that the above list mainly deals with 'impede dynamic analysis & tampering' but a similar enumeration of depth characteristics could be made for 'impeding comprehension' and others. This list is written from an attacker's pov, knowing what makes it more difficult and so what should fundamentally be done. E.g. for 'comprehension' I would start thinking about blockers for some manual 'reasoning' all the way to recent program synthesis tools.

[cpholguera]

hi @dzan, great proposal! we'll definitely consider it and come back with some feedback from our side. Thanks a lot and if there's anything else please let us know.

[cpholguera]

More details here:

https://drive.google.com/file/d/18-SMIDaNFPKHK8V6wWQqvj0UTu4toteQ/view?usp=sharing

[maizuka]

Could you tell me when v2 is planned to be released? Will you continue to refactor the rest chapters (ARCHITECTURE, AUTH, CODE, RESILIENCY) after PLATFORM?

[cpholguera]

Hello @maizuka, since this is a community and voluntary effort it's difficult to define a concrete release date. If everything goes well we'll keep the current pace of one category per month. The next category is MASVS-CODE and we'll cover the rest as well. After we close the last one we'll take some time to analyze the results as a whole and we might make some further optimizations.

You can stay up to date by following all our Twitter and LinkedIn. We'll keep posting in GitHub as well.

https://twitter.com/OWASP_MSTG

https://www.linkedin.com/mwlite/in/carlos-holguera

[maizuka]

Thank you!